Medium data display device, medium data display method, and medium data display program

ABSTRACT

The present invention provides a media data display device, a media data display method and a media data display program for displaying, on a display screen, multimedia data made up of a plurality of media including moving images in a variety of display methods. The media data display device ( 85 ) is a media data display device for displaying, on a display screen, multimedia data made up of a plurality of media including moving images, and is provided with a scaling determining portion ( 87 ), a scaling portion ( 89 ) and a display portion ( 19 ). The scaling determining portion ( 87 ) determines whether to display the moving images after changing their image size or to display the moving images without changing their image size. The scaling portion ( 89 ) and the display portion ( 19 ) are means for displaying the multimedia data, and display the moving images after switching whether to perform scaling or not based on a determination result from the scaling determining portion ( 87 ).

TECHNICAL FIELD

The present invention relates to media data display devices, media datadisplay methods and media data display programs, particularly to a mediadata display device, a media data display method and a media datadisplay program for displaying, on a display screen, multimedia datamade up of a plurality of media including moving images.

BACKGROUND ART

In recent years, techniques for distributing video, audio or text dataand the like over communications networks such as the Internet have beendeveloped. One example is a method in which digitized video, audio ortext data and the like is transmitted after being packetized inaccordance with a communication protocol such as RTP (Real TimeTransport Protocol) and HTTP/TCP (Hyper Text Transport Protocol,Transmission Control Protocol). Conventional methods (see, e.g.,JP2002-199370A (page 6)) related to distribution of video, audio or textdata and the like are known which use MPEG-4 as the audio and videoencoding method and an MP4 (ISO/IEC 14496-1:2001) file as the fileformat. In the case of RTP or TS (Transport Stream, used in digitalbroadcasting), an MP4 file is not transmitted as it is, but encoded dataitself is packetized and transmitted, whereas in the case of HTTP/TCP,an MP4 file is transmitted as it is.

Conventionally, at the time of displaying video multiplexed in an MP4file, the video is displayed based on the size of the display area andthe image size of the video included in the MP4 file. That is, when thesize of the video display area is different from the image size of thevideo, it is necessary to scale the video image size to the size of thevideo display area before displaying the video.

The structure of an MP4 file is described with reference to FIGS. 29 to32, together with a further description of the size of the video displayarea and the video image size.

FIG. 29 is a diagram illustrating the file structure of an MP4 file. TheMP4 file is a group of objects called “Boxes”. The boxes are arrangedhierarchically (see FIG. 29(a)), and each box is made up of [size],[type] and [data] (see FIG. 29(b)). [size] indicates the data size ofthe box. [type] stores four letters (e.g., ‘moov’ or ‘trak’, which willbe described later) for identifying the box. [data] stores datacorresponding to [type]. Additionally, many boxes further store[version] and [flags] (see FIG. 29(c)). With this data structure, evenwhen a box of an unrecognizable [type] is acquired, it is possible toadvance the processing by ignoring and skipping that box.

FIG. 30 is a diagram illustrating a method for storing data in the MP4file. The MP4 file contains File Type Box (hereinafter, referred to as‘ftyp’), Movie Box (hereinafter, referred to as ‘moov’) and Media DataBox (hereinafter, referred to as ‘mdat’).

‘ftyp’ is placed at the beginning of the MP4 file, and indicates anidentifier (e.g., [major_brand] or [compatible_brand]) indicating thespecification on which the MP4 file is based on. When the MP4 file isbased on the 3GPP (Third Generation Partnership Project) standard, forexample, [3gp4] or [3gp5] is used. ‘moov’ stores header information, anda single ‘moov’ is included in an MP4 file. Specifically, ‘moov’includes Movie Header Box (hereinafter, referred to as ‘mvhd’) (notshown) that stores information (e.g., the time scale and the duration)related to reproduction of media contained in the MP4 file, and TrackBox (hereinafter, referred to as ‘trak’) that stores header information(e.g., the frame size, the frame duration, the initializationinformation and the data storage area) for each medium. For example, inan MP4 file containing audio and video, an independent ‘trak’ is presentfor each medium, such as an audio “trak” and a video “trak”. ‘mdat’stores data of each medium in media reproduction units called samples(e.g., frames, in the case of video). The stored position of the samplesin ‘mdat’ is specified by ‘moov’. In the following, an MP4 file having astructure in which all the header information of various media is storedin ‘moov’ is described. Although a structure (called a fragment) inwhich media data is divided into plural sections and header informationis attached to each of the divided sections is also defined in MP4, itsdescription has been omitted here.

FIG. 31 is a diagram illustrating the structure of ‘trak’. ‘trak’includes Track Header Box (hereinafter, referred to as ‘tkhd’) and MediaBox (hereinafter, referred to as ‘mdia’).

‘tkhd’ has the size of the video display area (Wp, Hp) in the case ofvideo, or includes coordinates (Tx, Ty) of an upper left point of thetext display area that are expressed taking an upper left point of thevideo display area as a reference and the size of the text display (Wt,Ht) in the case of text. In this regard, FIG. 32 shows the positionalrelationship between a video display area 300 and a text display area301. The video display area 300 is set in a display screen 302. Thevideo display area 300 has the size (Wp, Hp). The text display area 301has the size (Wt, Ht) at the coordinates (Tx, Ty) taking the upper leftpoint of the video display area 300 as a reference.

‘mdia’ (see FIG. 31) includes Media Header Box (hereinafter, referred toas ‘mdhd’) (not shown), Handler Reference Box (hereinafter, referred toas ‘hdlr’) and Media Information Box (hereinafter, referred to as‘minf’). ‘mdhd’ stores information (e.g., the time scale and theduration) related to reproduction of media stored in ‘trak’. ‘hdlr’stores an identifier (handler_type such as video [vide], audio [soun]and text [text]) indicating the type of media stored in ‘trak’. At thetime of reproducing the MP4 file, the type of the media stored in ‘trak’is determined by referencing ‘hdlr’. ‘minf’ stores, for example, VideoMedia Header Box (hereinafter, referred to as ‘vmhd’) (not shown), DataInformation Box (hereinafter, referred to as ‘dinf’) (not shown) andSample Table Box (hereinafter, referred to as ‘stbl’). ‘vmhd’ includesbasic information related to video display. ‘dinf’ includes, forexample, Data Reference Box (hereinafter, referred to as ‘dref’) (notshown) indicating the position at which the information of media isstored in ‘trak’.

‘stbl’ includes, for example, Sample Size Box (hereinafter, referred toas ‘stsz’), Decoding Time to Sample Box (hereinafter, referred to as‘stts’), Sample Description Box (hereinafter, referred to as ‘stsd’),Sample To Chunk Box (hereinafter, referred to as ‘stsc’) (not shown) andChunk Offset Box (hereinafter, referred to as ‘stco’) (not shown).

‘stsz’ stores the size of each sample. ‘stts’ stores the reproductiontime length of each sample. ‘stsd’ stores sample reproductioninformation called Visual Sample Entry in the case of video. VisualSample Entry stores, for example, a video image size (Wv, Hv). ‘stsc’stores information that associates Visual Sample Entry with each sample.

FIG. 33 shows the overall data structure of the above-described MP4file. The size of the video display area (Wp, Hp) and the video imagesize (Wv, Hv) are stored as [data] of ‘tkhd’ and [data] of ‘stsd’,respectively.

A conventional scaling display for video is described with reference toFIGS. 34 and 35.

FIG. 34 is a block diagram illustrating the configuration of aconventional media data display device 10 for MP4 files. The media datadisplay device 10 is, for example, a device that downloads an MP4 filestored in a server by using HTTP and reproduces it, or a device thatreproduces an MP4 file stored in a recording medium such as an SD card,and is provided in televisions, personal computers (PCs), mobile phones,or other devices capable of displaying image signals. The media datadisplay device 10 includes a data acquisition portion 11, a headerseparation portion 12, a sample information acquisition portion 13, asample acquisition portion 14, a decoding portion 15, a frame memory 16,a basic display information acquisition portion 17, a scaling portion 18and a display portion 19.

The data acquisition portion 11 acquires an MP4 file. The headerseparation portion 12 separates, from the MP4 file, sample-relatedinformation such as ‘stts’, ‘stsc’, ‘stsz’ and ‘stco’ anddisplay-related information such as ‘tkhd’ and ‘stsd’. The sampleinformation acquisition portion 13 acquires sample-related informationsuch as ‘stts’, ‘stsc’, ‘stsz’ and ‘stco’. The sample acquisitionportion 14 acquires, for example, the size, the stored position, thedecoding and display time of the sample based on the sample-relatedinformation acquired by the sample information acquisition portion 13.Furthermore, it requests the sample data from the data acquisitionportion 11 based on the information acquired from the sample informationacquisition portion 13. The decoding portion 15 decodes the sample dataacquired by the sample acquisition portion 14 for each sample. The framememory 16 stores the decoded sample data, for example, for each frame.The basic display information acquisition portion 17 acquiresdisplay-related information such as ‘tkhd’ and ‘stsd’. Here, ‘tkhd’includes the size of the video display area (Wp, Hp), and ‘stsd’includes a video image size (Wv, Hv).

The scaling portion 18 acquires the size of the video display area (Wp,Hp) and the video image size (Wv, Hv) from the basic display informationacquisition portion 17. Furthermore, it scales frame data having theimage size (Wv, Hv) that is stored in the frame memory 16 to the displayarea size (Wp, Hp) in a fill mode. The display portion 19 acquires thesize of the video display area (Wp, Hp) and the scaled frame data fromthe scaling portion 18, and writes data for video display in a displaymemory (not shown) or the like.

FIG. 35 is a diagram illustrating scaling. When the image size (Wv, Hv)of frame data 305 stored in the frame memory 16 is different from thedisplay area size (Wp, Hp) acquired by the basic display informationacquisition portion 17, the frame data 305 is scaled to the videodisplay area 300 in a fill mode.

DISCLOSURE OF INVENTION

In the case of the conventional media data display device 10, video isdisplayed after being scaled to the size of the video display area (Wp,Hp) indicated by ‘tkhd’. This results in a uniform display, making itdifficult to realize a variety of display methods. Further, in the caseof the conventional media data display device 10, scaling needs to beperformed for several to several tens of times per second at the time ofdisplaying video, so that the load on the CPU and the electric powerconsumption are high, which is problematic especially for portableterminals requiring lower power consumption and lower throughput.

It is an object of the present invention to provide a media data displaydevice, a media data display method and a media data display program fordisplaying, on a display screen, multimedia data comprising a pluralityof media including moving images in a variety of display methods.

It is another object of the present invention to provide a media datadisplay device, a media data display method and a media data displayprogram that switch display methods according to the capability of theterminal, the conditions at use and the like.

A media data display device described in claim 1 is a media data displaydevice for displaying, on a display screen, multimedia data comprising aplurality of media including moving images, and comprises a scalingdetermining means and a media data display means. The scalingdetermining means determines whether to display the moving images afterchanging their image size or to display the moving images withoutchanging their image size. The media data display means is a means fordisplaying multimedia data, and displays the moving images afterswitching whether to perform scaling or not based on a determinationresult from the scaling determining means.

Here, “multimedia data” refers to, for example, an MP4 file in whichvideo, audio, text, still images or the like are multiplexed and stored(hereinafter, the same applies in this section).

According to the media data display device of the present invention, thescaling determining means determines whether to perform scaling display,so that the moving images are not uniformly scaled and displayed in afixed area, but are displayed in a size intended by the creator of themultimedia data. Additionally, it is also possible to display the movingimages without scaling them, in which case the processing load on thedevice can be reduced.

A media data display device described in claim 2 is the media datadisplay device according to claim 1, wherein the scaling determiningmeans makes the determination based on scaling determining informationincluded in the multimedia data.

According to the media data display device of the present invention, thescaling determining means determines whether to perform scaling based onthe scaling determining information. Here, the intention of the creatorof the multimedia data concerning scaling display is indicated in thescaling determining information, so that the moving images aredisplayed, reflecting the intention of the creator of the multimediadata.

A media data display device described in claim 3 is the media datadisplay device according to claim 2, wherein the scaling determininginformation is flag information indicating whether to perform scaling.

According to the media data display device of the present invention, thescaling determining means determines whether to perform scaling, basedon the flag information.

A media data display device described in claim 4 is the media datadisplay device according to claim 1, wherein the scaling determiningmeans makes the determination based on the remaining capacity of abattery with which the media data display device is provided. Thescaling determining means does not perform scaling display, for example,when the remaining capacity of the battery is low.

According to the media data display device of the present invention,whether to perform scaling is determined according to the remainingbattery capacity, so that electric power consumption can be reduced bynot performing scaling display, for example, when the remaining batterycapacity is low, thereby making it possible to extend the serviceabletime of the media data display device.

A media data display device described in claim 5 is the media datadisplay device according to claim 1, wherein the scaling determiningmeans includes a scaling setting means for setting whether to displaythe moving images after changing their image size or to display themoving images without changing their image size. Here, the scalingsetting means, for example, may directly specify whether to performscaling, or may indirectly specify whether to perform scaling by settingthe operations of the media data display device.

According to the media data display device of the present invention, thescaling determining means determines whether to perform scaling displayin accordance with the user's settings for the scaling setting means, sothat the moving images are displayed, reflecting the user's intentionconcerning scaling.

A media data display device described in claim 6 is the media datadisplay device according to claim 1, wherein the scaling determiningmeans makes the determination based on an attribute of the movingimages. Here, “attribute of the moving image” refers to, for example,the image size or the frame rate of the moving images.

According to the media data display device of the present invention, thescaling determining means determines not to perform scaling when theprocessing load on the media data display device is high, for example,when the frame rate is high. This prevents an excessive processing loadfrom being applied on the media data display device.

A media data display device described in claim 7 is a media data displaydevice for displaying, on a display screen, multimedia data comprising aplurality of media including moving images, and comprises a scalingmeans and a media data display means. The scaling means scales themoving images with respect to a moving image display area in which themoving images can be displayed in the display screen, based on scalingspecifying information that specifies a method for scaling the movingimages. The media data display means is a means for displaying themultimedia data, and displays the moving images scaled by the scalingmeans in the moving image display area.

According to the media data display device of the present invention, thescaling means scales the moving images in a specified scaling method, sothat the moving images are not uniformly scaled and displayed in a fixedarea, but are scaled in a variety of scaling methods and displayed.

A media data display device described in claim 8 is the media datadisplay device according to claim 7, wherein the scaling means obtainsthe scaling specifying information from the multimedia data.

Here, “scaling setting information” refers to, for example, flaginformation indicating the scaling method to be used or numberinformation indicating the scaling method to be used.

According to the media data display device of the present invention, thescaling means obtains the scaling specifying information from themultimedia data, so that the moving images are scaled and displayed,reflecting the intention of the creator of the multimedia data.

A media data display device described in claim 9 is the media datadisplay device according to claim 7 or 8, and further comprises ascaling determining means for determining whether to display the movingimages after changing their image size or to display the moving imageswithout changing their image size. The media data display means displaysthe moving images after switching whether to perform scaling or notbased on a determination result from the scaling determining means.

According to the media data display device of the present invention, thescaling determining means determines whether to perform scaling display,so that the moving images are not uniformly scaled and displayed in afixed area, but are displayed in a size intended by the creator of themultimedia data. Additionally, it is also possible to display the movingimages without scaling them, in which case the processing load on thedevice can be reduced.

A media data display device described in claim 10 is the media datadisplay device according to claim 7, and further comprises a scalingmethod setting means for setting the scaling specifying information.

According to the media data display device of the present invention, thescaling method is set by the scaling method setting means, so that themoving images are scaled and displayed, reflecting the intention of theuser of the media data display device.

A media data display method described in claim 11 is a media datadisplay method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises ascaling determining process and a media data display process. Thescaling determining process determines whether to display the movingimages after changing their image size or to display the moving imageswithout changing their image size. The media data display process is aprocess of displaying the multimedia data, and displays the movingimages based on a determination result from the scaling determiningprocess.

According to the media data display method of the present invention, thescaling determining process determines whether to perform scalingdisplay, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. Additionally, it is also possible todisplay the moving images without scaling them, in which case theprocessing load on the device can be reduced.

A media data display method described in claim 12 is a media datadisplay method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises ascaling process and a media data display process. The scaling processscales the moving images with respect to a moving image display area inwhich the moving images can be displayed in the display screen, based onscaling specifying information that specifies a method for scaling themoving images. The media data display process is a process of displayingthe multimedia data, and displays the moving images scaled by thescaling process in the moving image display area.

According to the media data display method of the present invention, thescaling process scales the moving images in a specified scaling method.Therefore, the moving images are not uniformly scaled and displayed in afixed area, but are scaled in a variety of scaling methods anddisplayed.

A media data display program described in claim 13 is a media datadisplay program for performing, with a computer, a media data displaymethod for displaying, on a display screen, multimedia data comprising aplurality of media including moving images, and the media data displayprogram lets the computer perform a media display method comprising ascaling determining process and a media data display process. Thescaling determining process determines whether to display the movingimages after changing their image size or to display the moving imageswithout changing their image size. The media data display process is aprocess of displaying the multimedia data, and displays the movingimages based on a determination result from the scaling determiningprocess.

According to the media data display program of the present invention,the scaling determining process determines whether to perform scalingdisplay, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. Additionally, it is also possible todisplay the moving images without scaling them, in which case theprocessing load on the device can be reduced.

A media data display program described in claim 14 is a media datadisplay program for performing, with a computer, a media data displaymethod for displaying, on a display screen, multimedia data comprising aplurality of media including moving images, and the media data displayprogram lets the computer perform a media data display method comprisinga scaling process and a media data display. The scaling process scalesthe moving images with respect to a moving image display area in whichthe moving images can be displayed in the display screen, based onscaling specifying information that specifies a method for scaling themoving images. The media data display process is a process of displayingthe multimedia data, and displays the moving images scaled by thescaling process in the moving image display area.

According to the media data display program of the present invention,the scaling process scales the moving images in a specified scalingmethod. Therefore, the moving images are not uniformly scaled anddisplayed in a fixed area, but are scaled in a variety of scalingmethods and displayed.

The present invention described above provides a media data displaydevice, a media data display method and a media data display program fordisplaying, on a display screen, multimedia data comprising a pluralityof media including moving images in a variety of display methods.

Furthermore, the present invention described above provides a media datadisplay device, a media data display method and a media data displayprogram that switch display methods according to the capability of theterminal, the conditions at use and the like.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing the configuration of a media datadisplay device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing the relationship between frame data and avideo display area in data for display (the first embodiment).

FIG. 3 is a flowchart showing a media data display method (the firstembodiment).

FIG. 4 is a block diagram showing the configuration of a media datadisplay device according to a second embodiment of the presentinvention.

FIG. 5 is a diagram showing a syntax of ‘stdc’ (the second embodiment).

FIG. 6 is a diagram showing the relationship between frame data and avideo display area in data for display (the second embodiment).

FIG. 7 is a flowchart showing a media data display method (the secondembodiment).

FIG. 8 is a diagram showing the relationship between frame data and avideo display area in data for display (the second embodiment).

FIG. 9 is a diagram showing the relationship between frame data and avideo display area in data for display (the second embodiment).

FIG. 10 is a block diagram showing the configuration of a media datadisplay device according to a modification of the second embodiment ofthe present invention.

FIG. 11 is a diagram showing the relationship between frame data and avideo display area in data for display (a modification of the secondembodiment).

FIG. 12 is a block diagram showing the configuration of a media datadisplay device according to a modification of the second embodiment (amodification of the second embodiment).

FIG. 13 is a diagram showing the relationship between frame data and avideo display area in data for display (a modification of the secondembodiment).

FIG. 14 is a block diagram showing the configuration of a media datadisplay device according to a third embodiment of the present invention.

FIG. 15 is a diagram showing a syntax of ‘stsl’ (the third embodiment).

FIG. 16 is a diagram showing various scaling methods (the thirdembodiment).

FIG. 17 is a diagram for illustrating an operation of a scaling portion(the third embodiment).

FIG. 18 is a diagram for illustrating an operation of a scaling portion(the third embodiment).

FIG. 19 is a flowchart showing a media data display method (the thirdembodiment).

FIG. 20 is a diagram showing a syntax of ‘stsl’ (the third embodiment).

FIG. 21 is a block diagram showing the configuration of a media datadisplay device according to a modification of the third embodiment.

FIG. 22 is a diagram showing a syntax of ‘stsl’ (the third embodiment).

FIG. 23 is a block diagram showing the overall configuration of acontent serving system (a fourth embodiment).

FIG. 24 shows an example of a mobile phone using a moving image encodingmethod and a moving image decoding method (the fourth embodiment).

FIG. 25 is a block diagram of the mobile phone (the fourth embodiment).

FIG. 26 shows an example of a digital broadcasting system (the fourthembodiment).

FIG. 27 is a diagram showing a syntax of ‘stsl’ (other considerations).

FIG. 28 is a diagram illustrating a method for acquiring display-relatedinformation from Visual Sample Entry (other considerations).

FIG. 29 shows the file structure of an MP4 file according to the priorart.

FIG. 30 shows a method for storing data in the MP4 file (the prior art).

FIG. 31 is the structure of Track Box (the prior art).

FIG. 32 is a diagram showing the positional relationship between a videodisplay area and a text display area (the prior art).

FIG. 33 shows the data structure of the MP4 file (the prior art).

FIG. 34 is a block diagram showing the configuration of a conventionalmedia data display device (the prior art).

FIG. 35 is a diagram illustrating scaling (the prior art).

BEST MODE FOR CARRYING OUT THE INVENTION

In the following embodiments, a media data display method using a mediadata display device is described. It should be noted that although thedisplay method is described taking video as an example in the followingembodiments, the same display method can also be applied to still imagesin JPEG (Joint Photographic Experts Group), for example.

First Embodiment

A first embodiment of the present invention is described with referenceto FIGS. 1 to 3.

FIG. 1 is a block diagram illustrating the configuration of a media datadisplay device 20 according to the first embodiment of the presentinvention. The media data display device 20 is, for example, a devicethat downloads an MP4 file (see FIGS. 29 to 33, which also applies tothe following “Embodiments of the Invention”) stored in a server byusing HTTP and reproduces it, or a device that reproduces an MP4 filestored in a recording medium such as an SD card, and is provided intelevisions, personal computers (PCs), mobile phones, or other devicescapable of displaying image signals.

In the case of the conventional media data display device 10, videocontained in the MP4 file is displayed after being scaled to the displayarea size (Wp, Hp) stored in ‘tkhd’, regardless of the image size (Wv,Hv) stored in ‘stsd’. On the other hand, the media data display device20 does not include the scaling portion 18 (see FIG. 34), and displaysvideo included in the MP4 file without scaling it.

Here, when the size of the video display area is freely set in the mediadata display device 20, the MP4 file stores a special value (e.g., (Wp,Hp)=(0, 0)) as the display area size stored in ‘tkhd’. That is, when theMP4 file has a special value as the display area size (Wp, Hp), itindicates that the size of the video display area is not specified inthe MP4 file.

(1) Configuration of Media Data Display Device 20

Those parts of the media data display device 20 shown in FIG. 1according to the present invention that are different from theconventional media data display device 10 shown in FIG. 34 aredescribed. It should be noted that parts that perform similar operationsas in the conventional media data display device 10 are given the samereference numerals, and their description has been omitted.

The media data display device 20 is different from the media datadisplay device 10 in that it includes a display area converting portion21 and a terminal display screen size acquisition portion 22. Theterminal display screen size acquisition portion 22 acquires the displayscreen size (Wd, Hd) of a display screen (not shown) that displays thedata for video display output from the display portion 19. The displayarea converting portion 21 acquires the display area size (Wp, Hp)stored in ‘tkhd’ from the basic display information acquisition portion17. Further, the display area converting portion 21 sets the videodisplay area based on the value of the display area size (Wp, Hp) andthe value of the display screen size (Wd, Hd). The display portion 19outputs data for display such that the video frame data stored in theframe memory 16 is displayed in the video display area set by thedisplay area converting portion 21.

(2) Operation of Display Area Converting Portion 21

Setting of the video display area by the display area converting portion21 is described in further detail.

If the display area size (Wp, Hp) acquired from the basic displayinformation acquisition portion 17 does not show the special value (0,0), the display area converting portion 21 sets the acquired displayarea size (Wp, Hp) as the size of the video display area.

On the other hand, if the display area size (Wp, Hp) acquired from thebasic display information acquisition portion 17 shows the special value(0, 0), the display area converting portion 21 sets the display screensize (Wd, Hd) as the size of the video display area.

The relationship between the frame data and the video display area inthe data for display is described with reference to FIG. 2. Frame data24 has the image size (Wv, Hv), and is displayed in a video display area25 without being scaled. The size of the video display area 25 is set toeither the display area size (Wp, Hp) or the display screen size (Wd,Hd) by the display area converting portion 21.

In FIG. 2, the frame data 24 is centered in the video display area 25.Here, the position of the frame data 24 relative to the video displayarea 25 is determined by the display portion 19.

(3) Media Data Display Method in Media Data Display Device 20

A media data display method in the media data display device 20 isdescribed with reference to FIG. 3.

The display area converting portion 21 acquires the display screen size(Wd, Hd) from the terminal display screen size acquisition portion 22(step S26). Further, the display area converting portion 21 acquires thedisplay area size (Wp, Hp) stored in ‘tkhd’ from the basic displayinformation acquisition portion 17 (step S27).

If the display area size (Wp, Hp) acquired from the basic displayinformation acquisition portion 17 does not show the special value (0,0) (step S28), the display area converting portion 21 sets the acquireddisplay area size (Wp, Hp) as the size of the video display area (stepS29). On the other hand, when the display area size (Wp, Hp) acquiredfrom the basic display information acquisition portion 17 shows thespecial value (0, 0) (step S28), the display area converting portion 21sets the display screen size (Wd, Hd) as the size of the video displayarea (step S30).

The display portion 19 outputs data for display such that the frame datastored in the frame memory 16 is displayed in the video display area,without being scaled (step S31).

(4) Effect of Media Data Display Device 20

In the case of the conventional media data display device 10, scalingneeds to be performed for several to several tens of times per second atthe time of displaying video, so that the load on the CPU and theelectric power consumption are high. On the other hand, in the case ofthe media data display device 20 according to the embodiment of thepresent invention, it is not necessary to perform scaling, thus makingit possible to reduce the throughput and the electric power consumption.This is effective especially in cases where the media data displaydevice 20 is provided in mobile phones and the like.

Furthermore, in the case of the media data display device 20, it ispossible to set the size of the video display area according to the sizeof the display screen if the acquired display area size (Wp, Hp) shows aspecial value. Accordingly, it is possible to display the video in anappropriate size.

In MPEG-4 AVC (Advanced Video Coding), the image size (Wv, Hv) of framedata changes in the middle of video. In the case of the media datadisplay device 20, the image size does not necessarily need to be scaledto (Wp, Hp) at the time of displaying such video, so that it is possibleto display the video according to the image size (Wv, Hv) of the framedata.

(5) Modifications of Media Data Display Device 20

The present invention is not limited to the above-described embodiment,and various modifications may be made thereto as long as they do notdepart from the spirit of the invention.

(5-1) First Modification of Display Area Converting Portion 21

(5-1-1)

If the display area size (Wp, Hp) acquired from the basic displayinformation acquisition portion 17 shows the special value (0, 0), thedisplay area converting portion 21 may set the size of the video displayarea after acquiring the display screen size (Wd, Hd) and the displayarea size of text, still images or the like included in ‘trak’.

In this case, it is possible to set the video display area, taking intoconsideration the display areas of other media.

(5-1-2)

If the display area size (Wp, Hp) acquired from the basic displayinformation acquisition portion 17 shows the special value (0, 0), thedisplay area converting portion 21 may set the size of the video displayarea according to the processing capability (the CPU frequency, thememory size, the data transmission speed and the bus speed) of the mediadata display device 20.

In this case, if there is enough processing capability left, it ispossible to display the video in a larger area.

(5-1-3)

If the display area size (Wp, Hp) acquired from the basic displayinformation acquisition portion 17 shows the special value (0, 0), thedisplay area converting portion 21 may set the video display area in apredetermined size. Specifically, the media data display device 20 mayfurther include a size setting portion that allows the user to set thesize of the video display area, and may set the video display area inthe set size, or may set the video display area in a size that has beenset according to the operation state of the media data display device20, such as a power saving mode.

In this case, it is possible to display the video as intended by theuser.

(5-2) Second Modification of Display Area Converting Portion 21

If the display area size (Wp, Hp) acquired from the basic displayinformation acquisition portion 17 does not show the special value (0,0) and if the display area size (Wp, Hp) is different from the displayscreen size (Wd, Hd), the display area converting portion 21 may set thesize of the video display area to the display screen size (Wd, Hd).

In this case, it is possible to always display the video in a sizeappropriate for the display screen.

(5-3) Modification of Display Portion 19

In the description of FIG. 2, it was stated that “the frame data 24 iscentered in the video display area 25”. Here, the frame data 24 does notnecessarily need to be centered.

In the description of FIG. 2, it was also stated “the position of theframe data 24 relative to the video display area 25 is determined by thedisplay portion 19”. Here, the media data display device 20 may includea display position determining portion that determines the displayposition of the frame data 24. In this case, the display portion 19acquires the display position from the display position determiningportion, and outputs data for display.

Second Embodiment

A second embodiment of the present invention is described with referenceto FIGS. 4 to 13.

FIG. 4 is a block diagram illustrating the configuration of a media datadisplay device 35 according to the second embodiment of the presentinvention. The media data display device 35 is, for example, a devicethat downloads an MP4 file stored in a server by using HTTP andreproduces it, or a device that reproduces an MP4 file stored in arecording medium such as an SD card, and is provided in televisions,personal computers (PCs), mobile phones, or other devices capable ofdisplaying image signals.

Similarly to the media data display device 20 described with referenceto FIG. 1, the media data display device 35 does not include the scalingportion 18 (see FIG. 34), and displays video included in an MP4 filewithout scaling it.

Here, the MP4 file acquired by the media data display device 35 stores,in ‘stbl’, Sample Display Center Box (hereinafter, referred to as‘stdc’) that specifies the area to be displayed with priority for eachvideo frame data.

FIG. 5 shows an example of the syntax of ‘stdc’. ‘stdc’ has the samestructure as that of the box described with reference to FIG. 29(b).Here, [entry_count] indicates the number of entries included in ‘stdc’.[sample_count] indicates the number of consecutive samples whose valuesof [display_center_x] and [display_center_y] are identical.[display_center_x] and [display_center_y] represent the X-coordinate andthe Y-coordinate of the center of the area to be displayed with priorityfor each sample (e.g., each frame data), respectively. Hereinafter, thecoordinates represented by (display_center_x, display_center_y) arereferred to as “priority display coordinates”, and described as (dcx,dcy). It should be noted that the method for specifying the priorityarea is not limited to this method, and it may also be specified by thecoordinates of the two apexes on the same diagonal line, for example. Itis also possible to specify the priority area by ranking the priority.Furthermore, it is also possible to classify image areas using an indexother than the priority, or to specify the priority area for largerunits, for example, for each chunk or track, each of which is a group ofa plurality of samples.

(1) Configuration of Media Data Display Device 35

Those parts of the media data display device 35 shown in FIG. 4according to the present invention that are different from theconventional media data display device 10 shown in FIG. 34 aredescribed. It should be noted that parts that perform similar operationsas in the conventional media data display device 10 are given the samereference numerals, and their description has been omitted.

The media data display device 20 is different from the media datadisplay device 10 in that it includes an expanded display informationacquisition portion 37 and a display position determining portion 38. Inaddition, a header separating portion 36 can recognize and separate‘stdc’ included in an MP4 file.

The expanded display information acquisition portion 37 acquires ‘stdc’separated by the header separating portion 36, and outputs prioritydisplay coordinates (dcx, dcy) for each frame data. The display positiondetermining portion 38 acquires the display area size (Wp, Hp) stored in‘tkhd’ and the image size (Wv, Hv) stored in ‘stsd’ from the basicdisplay information acquisition portion 17, and acquires the prioritydisplay coordinates (dcx, dcy) from the expanded display informationacquisition portion 37. The display position determining portion 38determines the display position of the frame data in the video displayarea such that the priority display coordinates (dcx, dcy) of the framedata having the image size (Wv, Hv) coincide with the center of thevideo display area having the display area size (Wp, Hp). The displayportion 19 outputs data for display such that the frame data stored inthe frame memory 16 is displayed at the display position determined bythe display position determining portion 38.

The relationship between frame data 45 and a video display area 46 inthe data for display is described with reference to FIG. 6. It should benoted that the display area size (Wp, Hp) is larger than the image size(Wv, Hv) in FIG. 6, so that the entire moving image is shown. The framedata 45 has the image size (Wv, Hv), and is displayed in the videodisplay area 46 without being scaled. The video display area 46 has thedisplay area size (Wp, Hp). The display position of the frame data 45 isdetermined such that the priority display coordinates (dcx, dcy)coincide with the center of the video display area.

(2) Media Data Display Method in Media Data Display Device 35

A media data display method in the media data display device 35 isdescribed with reference to FIG. 7.

The header separating portion 36 separates ‘tkhd’, ‘stsd’ and ‘stdc’from an MP4 file (step S48). The display position determining portion 38acquires the display area size (Wp, Hp) stored in ‘tkhd’ and the imagesize (Wv, Hv) stored in ‘stsd’ from the basic display informationacquisition portion, and acquires the priority display coordinates (dcx,dcy) stored in ‘stdc’ from the expanded display information acquisitionportion 37 (step S49). Furthermore, the display position determiningportion 38 determines the display position of the frame data in thevideo display area based on the acquired display area size (Wp, Hp),image size (Wv, Hv) and priority display coordinates (dcx, dcy) (stepS50). The display portion 19 outputs data for display such that theframe data stored in the frame memory 16 is displayed at the displayposition determined by the display position determining portion 38,without being scaled (step S51).

(3) Effect of Media Data Display Device 35

In the case of the conventional media data display device 10, scalingneeds to be performed for several to several tens of times per second atthe time of displaying video, so that the load on the CPU and theelectric power consumption are high. On the other hand, in the case ofthe media data display device 35 according to the embodiment of thepresent invention, scaling does not need to be performed, making itpossible to reduce the throughput and the electric power consumption.This is effective especially in cases where the media data displaydevice 35 is provided in mobile phones and the like.

Since the display position determining portion 38 determines the displayposition of moving images in the media data display device 35, themoving images can be displayed at an appropriate position by the displayportion 19.

Furthermore, with the media data display device 35, an important portioncan be displayed with priority for each sample (each frame data).Particularly, the area to be displayed with priority is changed for eachsample if the display area size (Wp, Hp) is smaller than image size (Wv,Hv), making it possible to realize a highly visually effective displaythat reflects the intention of the content provider. Additionally, thiseffect can be similarly achieved when the priority area is specified forlarger units, for example, for each chunk or track, each of which is agroup of a plurality of samples. Furthermore, when the priority area isspecified for each chunk or track, the processing load can be reduced ascompared with when it is specified for each sample.

In MPEG-4 AVC (Advanced Video Coding), the image size (Wv, Hv) of framedata changes in the middle of video. In the case of the media datadisplay device 35, such video can be displayed according to the imagesize (Wv, Hv) of the frame data.

The MP4 files that are taken as a basis of the present invention include‘stdc’, which has been newly defined. Therefore, when an MP4 fileincluding ‘stdc’ is acquired in the conventional media data displaydevice 10, which is not enabled for ‘stdc’, ‘stdc’ is not recognized andis skipped. That is, a data structure for an MP4 file is provided thatdoes not affect the display of the conventional media data displaydevice 10 and realizes a novel function for a device capable ofrecognizing ‘stdc’.

(4) Modifications of Media Data Display Device 35

(4-1) Modification of Display Position Determining Portion 38

(4-1-1)

In the case where the display position determining portion 38 determinesthe display position such that the priority display coordinates (dcx,dcy) coincide with the center of the video display area when the imagesize (Wv, Hv) is smaller than the display area size (Wp, Hp), it mayautomatically adjust the determined display position if there is enoughroom in the video display area.

The relationship between frame data 52 and a video display area 53 isdescribed with reference to FIG. 8. FIG. 8(a) shows the positionalrelationship between the frame data 52 and the video display area 53 inthe case where the display position is determined such that the prioritydisplay coordinates (dcx, dcy) of the frame data 52 coincide with thecenter of the video display area 53. In this case, the video displayarea 53 has a display margin 54, and the frame data 52 has a non-displayarea 55.

Therefore, the display position of the frame data 52 is adjusted suchthat the video display area 53 includes the priority display coordinates(dcx, dcy) and displays as large an area as possible of the frame data52. FIG. 8(b) shows the positional relationship between the adjustedframe data 52 and the video display area 53.

Another example of the relationship between frame data 52′ and a videodisplay area 53′ is described with reference to FIG. 9. It should benoted that although the image size (Wv, Hv) is smaller than the displayarea size (Wp, Hp) in the example shown in FIG. 8, the image size (Wv,Hv) is larger than the display area size (Wp, Hp) in this example. FIG.9(a) shows the positional relationship between the frame data 52′ andthe video display area 53′ in the case where the display position isdetermined such that the priority display coordinates (dcx, dcy) of theframe data 52′ coincide with the center of the video display area 53′.In this case, the video display area 53′ has a display margin 54′, andthe frame data 52′ has a non-display area 55′.

Therefore, the display position of the frame data 52′ is adjusted suchthat the video display area 53′ includes the priority displaycoordinates (dcx, dcy) and displays as large an area as possible of theframe data 52′. FIG. 9(b) shows the positional relationship between theadjusted frame data 52′ and the video display area 53′. In this case,although not all of the frame data 52′ is displayed, the prioritydisplay area is reliably displayed and a gray area 56′ is additionallydisplayed.

This makes it possible to effectively utilize the video display area andto display more information, in addition to realizing a highly visuallyeffective display that reflects the intention of the content provider.

(4-1-2)

The display position determining portion 38 may obtain the display areasize (Wp, Hp) stored in ‘tkhd’ and the image size (Wv, Hv) stored in‘stsd’ from the basic display information acquisition portion 17, andmay skip ‘stdc’ when the display area size (Wp, Hp) is larger than theimage size (Wv, Hv). In that case, display is performed in accordancewith the operation described in (4-2) below.

(4-2) Determination of Display Position in the Case of not Using ‘stdc’

When the MP4 file does not include ‘stdc’, or when the media displaydevice is a device that cannot recognize ‘stdc’, it is necessary todetermine the display position of the video in the video display area.

(4-2-1)

The operation of the media data display device 35 when the MP4 file doesnot include ‘stdc’ is described below.

When the display position determining portion 38 cannot acquire thepriority display coordinates (dcx, dcy) for each frame data from theexpanded display information acquisition portion 37, it determines thedisplay position of the frame data such that a predetermined point(e.g., the upper left, the upper right, the lower left, the lower rightor the center) of the video display area coincides with a predeterminedpoint (e.g., the upper left, the upper right, the lower left, the lowerright or the center) of the frame data.

This makes it possible to determine the display position of the framedata even when the MP4 file does not include ‘stdc’.

(4-2-2)

Further, when the media data display device cannot interpret ‘stdc’, thedisplay position of the frame data may be determined in accordance witha user instruction.

<1>

FIG. 10 is a block diagram illustrating the configuration of a mediadata display device 56 according to a modification of the secondembodiment. Similarly to the media data display device 20 described withreference to FIG. 1, the media data display device 56 does not includethe scaling portion 18 (see FIG. 34), and displays video included in anMP4 file without scaling it.

Those parts of the media data display device 56 shown in FIG. 10 thatare different from the conventional media data display device 10 shownin FIG. 34 are described. It should be noted that parts that performsimilar operations as in the conventional media data display device 10are given the same reference numerals, and their description has beenomitted.

The media data display device 56 is different from the media datadisplay device 10 in that it includes a display position setting portion57 and a display position determining portion 58.

The display position setting portion 57 allows the user to set thedisplay position of the frame data. For example, it allows the user toselect the display position from [center, top justify, bottom justify,right justify, left justify] and the like. The display positiondetermining portion 58 acquires the display area size (Wp, Hp) stored in‘tkhd’ and the image size (Wv, Hv) stored in ‘stsd’ from the basicdisplay information acquisition portion 17. Furthermore, it determinesthe display position of the frame data in the video display area basedon the setting made by the display position setting portion 57. Thedisplay portion 19 outputs data for display such that the frame datastored in the frame memory 16 is displayed at the display positiondetermined by the display position determining portion 58.

<2>

The relationship between frame data 61 and a video display area 62 inthe data for display is described with reference to FIG. 11(a). Theframe data 61 has the image size (Wv, Hv), and is displayed in the videodisplay area 62 without being scaled. The video display area 62 has thedisplay area size (Wp, Hp). When [top justify] is selected with thedisplay position setting portion 57, the display position is determinedsuch that the top side of the frame data 61 coincides with the top sideof the video display area. In addition, the horizontal position isdetermined such that the frame data 61 is centered with respect to thevideo display area 62.

It should be noted that FIG. 11(a) shows a case where the display areasize (Wp, Hp) is larger than the image size (Wv, Hv). If the displayarea size (Wp, Hp) is smaller than the image size (Wv, Hv), only aportion of frame data 63 that is included in a video display area 64 isdisplayed. FIG. 11(b) shows the relationship between the frame data 63and the video display area 64 for the case where [top justify] isselected with the display position setting portion 57.

<3>

With this media data display device 56, it is possible to realize a moreeffective display that reflects the user's intention.

<4>

Additionally, the display position setting portion 57 may allow the userto input display position data (e.g., the layout and the size), inaddition to allowing the user to select the display position from[center, top justify, bottom justify, right justify, left justify] andthe like.

(4-2-3)

The media data display device may also determine the display position offrame data based on the display position of media (e.g. text and stillimages) other than video that are included in the MP4 file.

It should be noted that display may be performed in accordance with auser instruction or a predetermined display method, even when the mediadata display device can interpret ‘stdc’.

<1>

FIG. 12 is a block diagram illustrating the configuration of a mediadata display device 70 according to a modification of the secondembodiment. Similarly to the media data display device 20 described withreference to FIG. 1, the media data display device 70 does not includethe scaling portion 18 (see FIG. 34), and displays video included in anMP4 file without scaling it.

Those parts of the media data display device 70 shown in FIG. 12 thatare different from the conventional media data display device 10 shownin FIG. 34 are described. It should be noted that parts that performsimilar operations as in the conventional media data display device 10are given the same reference numerals, and their description has beenomitted.

The media data display device 70 is different from the media datadisplay device 10 in that it includes a display position determiningportion 71.

The display position determining portion 71 acquires the display areasize (Wp, Hp) stored in ‘tkhd’ of the video and the image size (Wv, Hv)stored in ‘stsd’ of the video from the basic display informationacquisition portion 17. Furthermore, it acquires the display position(Tx, Ty) of the text display area and the display area size (Wt, Ht) ofthe text display area that are stored in ‘tkhd’ of the text. Inaddition, the display position determining portion 71 determines thedisplay position of the frame data in the video display area based onthe acquired information. Specifically, it determines the position inthe vertical direction such that the upper end of the text display areacoincides with the lower end of the frame data, and determines theposition in the horizontal direction such that the center of the textdisplay area coincides with the center of the frame data. The displayportion 19 outputs data for display such that the frame data stored inthe frame memory 16 is displayed at the video display positiondetermined by the display position determining portion 71.

The positional relationship between frame data 76, a video display area77 and a text display area 78 in the data for display is described withreference to FIG. 13. The video display area 77 having the display areasize (Wp, Hp) is set in a display screen 79. Additionally, the textdisplay area 78 having the display area size (Wt, Ht) at the coordinates(Tx, Ty) taking the point at the upper left of the video display area 77as a reference is set.

Here, the display position in the vertical direction of the frame data76 having the image size (Wv, Hv) is determined so as to be above andadjacent to the text display area 78. Further, the display position inthe horizontal direction of the frame data 76 is determined such thatthe position in the horizontal direction of the center of the frame data76 coincides with that of the center of the text display area 78.

<2>

In the case of the media data display device 70, the display position ofthe frame data is determined based on the display positions of media(e.g., text and still images) other than video that are included in theMP4 file. Accordingly, each medium is displayed at an appropriateposition, improving the visibility of information display.

It should be noted that although the frame data was described as beingdisplayed so as to be above and adjacent to the text display area, theeffect of the present invention is not particularly limited to theupward direction, and can also be achieved when it is below and adjacentor adjacent in other directions.

In addition, the display position of the frame data may also bedetermined based not only on the display position (Tx, Ty) of the textdisplay area and the display area size (Wt, Ht) of the text display areathat are stored in ‘tkhd’ of text, but on the display position of a textsample in the text display area as well.

Third Embodiment

A third embodiment of the present invention is described with referenceto FIGS. 14 to 22.

FIG. 14 is a block diagram illustrating the configuration of a mediadata display device 85 according to the third embodiment of the presentinvention. The media data display device 85 is, for example, a devicethat downloads an MP4 file stored in a server by using HTTP andreproduces it, or a device that reproduces an MP4 file stored in arecording medium such as an SD card, and is provided in televisions,personal computers (PCs), mobile phones, or other devices capable ofdisplaying image signals.

The media data display device 85 determines whether to scale video framedata based on information acquired from the MP4 file.

Here, the MP4 file acquired by the media data display device 85 mayinclude ‘stdc’, which has been described in “Second embodiment” withreference to FIG. 5. In addition, the MP4 file acquired by the mediadata display device 85 may include, in ‘stbl’, Sample Scale Box(hereinafter, referred to as ‘stsl’), which specifies the scaling methodfor each sample (each frame data). It should be noted that specificationof the scaling method is not limited to being performed for each sample,and may be performed for larger units, for example, for each chunk ortrack, each of which is a group of a plurality of samples. In this case,the processing load can be reduced as compared with when the scalingmethod is specified for each sample.

FIG. 15 shows an example of the syntax of ‘stsl’. ‘stsl’ has the samestructure as that of the box described with reference to FIG. 29(b).Here, if [fill_flag] is set, then scaling in a fill mode is specified.If [hidden_flag] is set, then scaling in a hidden mode is specified. If[meet_flag] is set, then scaling in a meet mode is specified. When[slice_x_flag] is set, then scaling in a slice mode on the X-axis (inthe horizontal direction of the screen) is specified. If [slice_y_flag]is set, then scaling in a slice mode on the Y-axis (in the verticaldirection of the screen) is specified. It should be noted that at mostone of [fill_flag], [hidden_flag], [meet_x_flag], [meet_y_flag],[slice_x_flag] and [slice_y_flag] is always set in ‘stsl’.

Each of the scaling methods is further described with reference to FIG.16.

The fill mode is a mode in which frame data (see FIG. 16(a)) having theimage size (Wv, Hv) is displayed after being scaled to the display areasize (Wp, Hp) (see FIG. 16(b)).

The hidden mode is a mode in which a portion of frame data (see FIG.16(a)) having the image size (Wv, Hv) that can be displayed within avideo display area having the display area size (Wp, Hp) is displayed,without being scaled (see FIG. 16(c)). More specifically, in the case of(Wv<Wp) or (Hv<Hp), a specific background color is displayed in aportion of the video display area in which the frame data is notdisplayed. On the other hand, in the case of (Wv>Wp) or (Hv>Hp), theframe data that protrudes beyond (Wp, Hp) is clipped.

The meet mode is a mode in which the frame data is displayed after beingscaled until it is equal to the video display area in one of thevertical and horizontal dimensions, while maintaining the aspect ratio(the ratio of the vertical and horizontal dimensions [Wv:Hv] of theimage). A specific background color is displayed in a portion of thevideo display area in which the frame data is not displayed. In thismode, the frame data will not be clipped.

In the slice mode, the frame data is displayed after being scaled untilit is equal to the video display area in the vertical or horizontaldimension, while maintaining the aspect ratio (the ratio of the verticaland horizontal dimensions [Wv:Hv] of the image). A specific backgroundcolor is displayed in a portion of the video display area in which theframe data is not displayed. In this mode, the frame data may beclipped.

(1) Configuration of Media Data Display Device 85

Those parts of a media data display device 85 shown in FIG. 14 accordingto the present invention that are different from the conventional mediadata display device 10 shown in FIG. 34 are described. It should benoted that parts that perform similar operations as in the conventionalmedia data display device 10 are given the same reference numerals, andtheir description has been omitted.

The media data display device 85 is different from the media datadisplay device 10 in that it includes an expanded display informationacquisition portion 86 and a scaling determining portion 87.Furthermore, a header separating portion 88 is capable of recognizingand separating ‘stdc’ and ‘stsl’ included in the MP4 file. In addition,a scaling portion 89 is capable of performing scaling based on adetermination result from the scaling determining portion 87.

The expanded display information acquisition portion 86 acquires ‘stdc’and ‘stsl’ separated by the header separating portion 88. The scalingdetermining portion 87 determines to perform scaling if the expandeddisplay information acquisition portion 86 acquires ‘stsl’. The scalingportion 89 acquires the display area size (Wp, Hp) stored in ‘tkhd’ andthe image size (Wv, Hv) stored in ‘stsd’ from the basic displayinformation acquisition portion 17, and acquires the priority displaycoordinates (dcx, dcy) (see FIG. 5) stored in ‘stdc’ and the scalingmethod stored in ‘stsl’ from the expanded display informationacquisition portion 37. Furthermore, the scaling portion 89 acquires thedetermination result from the scaling determining portion 87, andperforms scaling if the determination result is positive. The scaledframe data is output to the display portion 19, together with thedisplay area size (Wp, Hp) of the frame data.

(2) Operation of Scaling Portion 89

The scaling operation of the scaling portion 89 is further described.The scaling portion 89 performs scaling based on the priority displayarea information stored in ‘stdc’. Specifically, ‘stdc’ stores thepriority display coordinates (dcx, dcy), which are the coordinates ofthe central point of the priority display area. Accordingly, the scalingportion 89 estimates the priority display area from the priority displaycoordinates (dcx, dcy), and automatically adjusts the display positionsuch that the priority display area is displayed in a video display areahaving the display area size (Wp, Hp). More specifically, the scalingportion 89 selects an appropriate scaling method such that no portion ofthe priority area will be omitted.

This will be more specifically described with reference to FIG. 17.Here, the priority display coordinates (dcx, dcy) are acquired for framedata 90 having the image size (Wv, Hv) (see FIG. 17(a)). The scalingportion 89 takes, as a priority display area 91, an area centering onthe priority display coordinates (dcx, dcy) that is maximal in the framedata 90 (see FIG. 17(b)). In this case, the priority display area 91 islocated at the lower left portion of the frame data 90. If the scalingmethod acquired by the scaling portion 89 is the fill mode, the scalingportion 89 scales the priority display area 91 in the slice mode, forexample, in the x direction to the same size as the display area size(Wp, Hp) (see FIG. 17(c)). Consequently, the priority display area 91 isdisplayed.

Further, another example is described with reference to FIG. 18. Here,the priority display coordinates (dcx, dcy) are acquired for frame data90′ having the image size (Wv, Hv) (see FIG. 18(a)). The scaling portion89 takes, as a priority display area 91′, an area centering on thepriority display coordinates (dcx, dcy) that is maximal in the framedata 90′ (see FIG. 18(b)). In this case, the priority display area 91′is an upper left portion of the frame data 90′. In this example, theaspect ratio of the image size (Wv, Hv) is different from that of thedisplay area size (Wp, Hp) (i.e., Wv is shorter than Hv, in contrast tothe above-described example), and, moreover, the priority display area91′ is located at the upper left of the frame data 90′. Therefore, ifthe slice mode is used in this embodiment and scaling is performed inthe slice mode in the x direction, a portion of the priority displayarea 91′ will be omitted when displayed.

Accordingly, the fill mode is selected as the scaling method, and as aresult, the scaling portion 89 scales the priority display area 91′ inthe fill mode to the same size as the display area size (Wp, Hp) (seeFIG. 18(c)). Thus, the scaling portion 89 estimates the priority displayarea from the priority display coordinates (dcx, dcy), and selects anappropriate scaling method such that the priority display area 91′ isdisplayed in a video display area having the display area size (Wp, Hp),so that the priority display area 91′ can be reliably displayed.

In addition, when ‘stdc’ is not present, scaling is performed, taking asa reference, a state in which the upper left of the image is made tocoincide with the upper left of the display area. Here, a state in whichthe lower left, the upper right, the lower right or the center of theimage is made to coincide with the lower left, the upper right, thelower right or the center of the display area may be taken as thereference for scaling, or the reference state may be set by the user.

(3) Media Data Display Method in Media Data Display Device 85

A media data display method in the media data display device 85 isdescribed with reference to FIG. 19.

The header separating portion 88 separates ‘tkhd’, ‘stsd’, ‘stdc’ and‘stsl’ from the MP4 file (step S95). The scaling portion 89 acquires thedisplay area size (Wp, Hp) stored in ‘tkhd’ and the image size (Wv, Hv)stored in ‘stsd’ from the basic display information acquisition portion17, and acquires the priority display coordinates (dcx, dcy) stored in‘stdc’ and the scaling method stored in ‘stsl’ from the expanded displayinformation acquisition portion 37 (step S96). If the expanded displayinformation acquisition portion 86 acquires ‘stsl’, the scalingdetermining portion 87 determines to perform scaling (step S97). If itis determined to perform scaling, the scaling portion 89 outputs thepriority display area of the frame data stored in the frame memory 16after scaling it to the display area size (Wp, Hp) (step S98). If it isdetermined not to perform scaling, the scaling portion 89 outputs thepriority display area of the frame data stored in the frame memory 16without scaling it (step S99).

Here, even when ‘stsl’ is present, scaling is not performed if thehidden mode is set as the scaling mode. In addition, when ‘stsl’ ispresent in the MP4 file, but the terminal cannot interpret ‘stsl’,scaling may be performed in the fill mode to the display area size (Wp,Hp) indicated by ‘tkhd’.

In the foregoing, scaling is always performed before display, inaccordance with the scaling operation indicated by ‘stsl’ when ‘stsl’ ispresent; however, it is also possible to additionally provideinformation for allowing the terminal side to select whether to followthe scaling operation of ‘stsl’. FIG. 20 shows an example of the syntaxfor setting this information in ‘stsl’. Here, if [constraint_flag] isset, it indicates that scaling is always performed in accordance withthe scaling operation indicated by ‘stsl’. If it is not set, itindicates that whether to perform scaling can be selected at theterminal. Accordingly, if [constraint_flag] is not set, whether toperform scaling may be determined based on the remaining capacity of thebattery of the terminal, or information such as the content of theuser's settings.

Further, by always performing scaling in the fill mode to the displayarea size (Wp, Hp) when ‘stsl’ is present, ‘stdc’ can be used only when‘stsl’ is present. Therefore, the MP4 file acquired by the media datadisplay device 85 may be allowed to store ‘stdc’ only when ‘stsl’ ispresent.

By performing this operation, it is possible to provide an expandedfunction relating to the display operation for a terminal capable ofinterpreting ‘stsl’ and ‘stdc’, while maintaining backward compatibilitywith conventional display operations performed when reproducing MP4files.

(4) Effect of Media Data Display Device 85

With the media data display device 85, it is possible to determinewhether to perform scaling based on ‘stsl’. Here, ‘stsl’ is a boxindicating the scaling method. Therefore, when ‘stsl’ is present, it isconceivable that the provider of the MP4 file intends the frame data tobe scaled before being displayed. That is, it is possible to realize adisplay that reflects the intention of the provider of the MP4 file.

In the case of the conventional media data display device 10, scalingneeds to be performed for several to several tens of times per second atthe time of displaying video, so that the load on the CPU and theelectric power consumption are high. On the other hand, in the case ofthe media data display device 85, scaling does not necessarily need tobe performed, making it possible to reduce the throughput and theelectric power consumption. This is effective especially in cases wherethe media data display device 85 is provided in mobile phones and thelike.

Additionally, the MP4 files that are taken as a basis of the presentinvention include ‘stsl’, which has been newly defined. Therefore, whenan MP4 file including ‘stsl’ is acquired in the conventional media datadisplay device 10, which is not enabled for ‘stsl’, ‘stsl’ is notrecognized and thus is skipped. That is, a data structure for an MP4file is provided that does not affect the display of the conventionalmedia data display device 10 and realizes a novel function for a devicecapable of recognizing ‘stsl’.

(5) Modifications of Media Data Display Device 85

(5-1) Modifications of Media Data Display Device 85

(5-1-1)

The media data display device 85 may not include the scaling determiningportion 87, and may always perform scaling before display. Here, thescaling portion always performs an operation similar to that performedby the scaling portion 89 when it is determined that scaling isperformed before display.

Additionally, when the expanded display information acquisition portion86 does not acquire ‘stsl’ and ‘stdc’, the scaling portion 89 mayperform scaling in the meet mode or the slice mode, instead of the fillmode.

Furthermore, when scaling is performed in the fill mode, the meet modeor the slice mode, scaling may be performed only by an integral scalingfactor such as ½, ¼, 2 and 3.

For example, this allows each two adjacent pixels to be sequentiallyprocessed when scaling is performed by a factor of ½, and thussimplifies processing during scaling, thereby saving electric powerconsumption.

(5-1-2)

The media data display device 85 may not include the expanded displayinformation acquisition portion 86 and the scaling determining portion87, and may determine whether to perform scaling and the scaling methodin accordance with a user instruction.

FIG. 21 is a block diagram illustrating the configuration of a mediadata display device 100 according to a modification of the thirdembodiment. Those parts of the media data display device 100 shown inFIG. 21 that are different from the media data display device 85 shownin FIG. 14 are described. It should be noted that parts that performsimilar operations as in the media data display device 85 are given thesame reference numerals, and their description has been omitted.

The media data display device 100 is different from the media datadisplay device 85 in that it does not include the expanded displayinformation acquisition portion 86 and the scaling determining portion87 that are included in the media data display device 85, and that itincludes a scaling method setting portion 101. The scaling methodsetting portion 101 allows the user to set the scaling method (e.g., thefill mode, the meet mode or the slice mode). In addition, a scalingportion 102 scales frame data having the image size (Wv, Hv) that isstored in the frame memory 16 to a video display area having the displayarea size (Wp, Hp) by the scaling method set by the scaling methodsetting portion 101.

This makes it possible to reflect the user's intention on whether toperform scaling and on the scaling method.

(5-2) Modifications of Scaling Determining Portion 87

(5-2-1)

The scaling determining portion 87 may determine whether to performscaling in accordance with the remaining capacity of the battery withwhich the media data display device is provided, instead of, or inaddition to determining whether to perform scaling based on the factthat the expanded display information acquisition portion 86 hasacquired ‘stsl’. That is, the scaling determining portion 87 acquiresthe remaining capacity of the battery from a battery management portion(not shown) to determine whether to perform scaling.

By prioritizing the determination of whether to perform scaling based onthe remaining battery capacity when the remaining capacity of thebattery is low, it is possible to reduce energy consumed by performingscaling, thus extending the serviceable time of the media data displaydevice 85.

When the remaining capacity of the battery is high, on the other hand,it is also possible to perform scaling before display, regardless ofwhether ‘stsl’ has been acquired or not.

(5-2-2)

The scaling determining portion 87 may switch whether to perform scalingor not in accordance with the user's settings, instead of, or inaddition to determining whether to perform scaling based on the factthat the expanded display information acquisition portion 86 hasacquired ‘stsl’. That is, the scaling determining portion 87 switcheswhether to perform scaling or not in accordance with a setting from ascaling setting portion (not shown).

Specifically, the user may specify whether to perform scaling with“scaling” menu on a menu screen of a portable terminal or the like thatincorporates the media data display device, or may determine not toperform scaling if a power saving mode is selected in “power savingsetting” menu.

Additionally, it is also possible to always prioritize the user'ssetting, regardless of whether ‘stsl’ has been acquired or not.

(5-2-3)

The scaling determining portion 87 may determine whether to performscaling based on information other than ‘stsl’ that is acquired from theMP4 file.

<1>

The scaling determining portion 87 may not perform scaling when theexpanded display information acquisition portion 86 acquires ‘stdc’, butnot ‘stsl’.

<2>

The scaling determining portion 87 may determine whether to performscaling based on ‘ftyp’ acquired by the basic display informationacquisition portion 17. For example, it may determine whether to performscaling based on whether an identifier (e.g., [major_brand] or[compatible_brand]) indicating a particular specification is included ornot.

<3>

The scaling determining portion 87 may determine whether to performscaling based on ‘mvhd’ or ‘tkhd’ acquired by the basic displayinformation acquisition portion 17.

For example, it may determine the frame rate of the frame data based onthe duration of ‘tkhd’ and the total number of samples in a trackacquired from a box such as ‘stsz’ included in ‘stbl’, or the sampleduration indicated in ‘stts’, and may perform scaling when the framerate is equal to or lower than a set value (e.g., a frame rate of 0.5frame/s versus the set value of 1 frame/s).

Alternatively, it may perform scaling when the time required forencoding and scaling one frame is shorter than the time the media datadisplay device 85 can allocate for decoding and displaying one frame(e.g., 1/15 second, in the case of 15 frame/s).

In this case, it is possible to switch whether to perform scaling or notin accordance with the throughput of the media data display device 85.

(6) Other Considerations

(6-1)

It has been described that, in an MP4 file, ‘stsl’ is stored in ‘stbl’and specifies the scaling method for each sample. Here, ‘stsl’ may bestored in ‘trak’ and may specify the same scaling method for each track.FIG. 22 shows an example of the syntax for indicating the scaling methodfor each track. At this time, it is preferable that ‘stsl’ is placeddirectly below ‘trak’. Alternatively, the scaling method may bespecified for each chunk, which is a group of a plurality of samples.

(6-2)

‘stdc’ and ‘stsl’ described in the above embodiment may be informationitems included in the same box. For example, in an MP4 file acquired bythe media data display device 85, the priority area information storedin ‘stdc’ may be included in ‘stsl’, and vice versa.

(6-3)

In the syntaxes shown in FIG. 15 and FIG. 20, the scaling method isspecified as flag information.

Here, the scaling method may be specified, for example, with the valueof [scale_method], which is represented by 8 bits. More specifically,one of values of [1] to [5] is given to each of the fill mode, thehidden mode, the meet mode, the slice mode on the X-axis (in thehorizontal direction of the screen), and the slice mode on the Y-axis(in the horizontal direction of the screen).

Any information for specifying the scaling method may be used, withoutlimitation to the above.

Furthermore, in this case, the rest of the values of [scale_method], forexample, the values of 128 to 255 may be used as a user specified areain the following manner.

(6-3-1)

The user specified area may be used for indicating information on thescaling method or the priority area that is other than the informationon the scaling method or the priority area described in the aboveembodiment.

<1>

For example, it may be used for indicating a standard specification suchas 3GPP (3rd Generation Partnership Project), or information on thescaling method or the priority area prescribed by the operationalstandard of the service provider. In this case, the standardspecification or a unique number assigned to the operational standard isstored in the user specified area.

<2>

It may also be used for indicating, for example, the priority duringdisplaying an image of a track. Specifically, the priority is indicatedwith specific bits (e.g., the first 3 bits) of [scale_method].Accordingly, for example, when the priority of a text track is 2 and thepriority of a video track is 1, it is possible, for example, to displaythe text with priority through overlaying or the like.

<3>

Furthermore, for example, when an MP4 file is transmitted as anauxiliary data for a TV program or for the main content such as adownload/streaming content, the user specified area may also be used forindicating that the MP4 file is an auxiliary data. Specifically, it ispossible to change the display method for the auxiliary data and themain content, for example, by displaying the auxiliary data by apredetermined method.

(6-3-2)

The user specified area may also be used for indicating the followinginformation.

<1>

For example, it may be used for indicating information related toscrolling. Specifically, “information related to scrolling” refers toinformation that includes information indicating a default (unscrolled)display area, or flag information indicating whether to enable ascrolling function when the terminal has a scrolling function and is notcapable of displaying the entire image area at once. Accordingly,through scrolling, it is possible, for example, to prevent an image tobe displayed from being scaled and displayed inappropriately. In otherwords, it is possible to appropriately display an image to be displayedthrough scrolling.

<2>

The user specified area may also be used for indicating, for example, aplurality of display-related information items associated with thecapability of the terminal that displays multi-media data. Here,“display-related information” refers to information related to displayof samples, including, for example, information stored in ‘stbl’. Morespecifically, “display-related information” refers to informationincluded in Visual Sample Entry, the above-described information storedin ‘stsl’, or information included in ‘stdc’.

The user specified area may indicate, for example, a plurality ofdisplay-related information items, using a plurality of ‘stsl’sassociated with the capability of the terminal, or may indicate aplurality of display-related information items associated with thecapability of the terminal by including a plurality of entries in‘stsl’. In the case of using a plurality of ‘stsl’s, a plurality ofVisual Sample Entries in each of which one ‘stsl’ is stored may beprovided.

The display-related information and the capability of the terminal areassociated, for example, by attaching an identifier for the terminal'scapability, such as High, Medium or Low, to each display-relatedinformation item. In the terminal that performs display, display isperformed based on information corresponding to High when the capabilityis high, and information corresponding to Low when the capability islow.

Accordingly, it is possible to acquire display-related informationcorresponding to the capability of the terminal, thus making itpossible, for example, to perform display by an optimum display methodin accordance with the capability of the receiving terminal thatdisplays multi-media data.

Further, when the same MP4 file is simultaneously viewed at a pluralityof terminals, display is performed based on the display-relatedinformation corresponding to the terminal with the lowest capability. Inthis case, it is possible to perform an identical display in all of aplurality of the terminals having different capabilities. Additionally,the capability of each of the terminals is exchanged by a predeterminedcommunication among the terminals.

<3>

The user specified area may also be used for indicating information thatspecifies, for example, the positional relationship, the layer and thescaling method at the time of simultaneously displaying the main contentand the auxiliary data (see <3> of (6-3-1) above).

Fourth Embodiment

An example of the application of the media data display device, themedia data display method and the media data display program, as well asa system using the same according to a fourth example of the presentinvention are described with reference to FIGS. 23 to 26.

FIG. 23 is a block diagram showing the overall configuration of acontent serving system ex 100 that realizes a content distributionservice. The provision area of the communications service is dividedinto desired sizes, and base stations ex 107 to ex 110, which are fixedradio stations, are provided in the respective cells.

The content serving system ex 100 is connected, for example, to theInternet ex 101 via an internet service provider ex 102 and a telephonenetwork ex 104, and to various devices such as a computer ex 111, a PDA(personal digital assistant) ex 112, a camera ex 113, a mobile phone ex114 and a camera-equipped mobile phone ex 115 via the base stations ex107 to ex 110.

The content serving system ex 100, however, is not limited to acombination as shown in FIG. 23, and it is also possible to connect aportion of the combination. Alternatively, each of the devices may bedirectly connected to the telephone network ex 104, without using thebase stations ex 107 to ex 110, which are fixed radio stations.

The camera ex 113 is a device capable of capturing moving images, suchas a digital video camera. The mobile phone is, for example, a mobilephone of the PDC (Personal Digital Communications) format, the CDMA(Code Division Multiple Access) format, the W-CDMA (Wideband-CodeDivision Multiple Access) format or the GSM (Global System for MobileCommunications) format, or a PHS (Personal Handyphone System), and anyof these may be used.

In addition, a streaming server ex 103 is connected to the camera ex 113via the base station ex 109 and the telephone network ex 104, storesencoded data captured with the camera ex 113 as an MP4 file, andtransmits that file. Alternatively, it is also possible to encode andmultiplex moving images captured with the camera-equipped mobile phone,and to transmit the multiplexed data to the server ex 103. Additionally,the captured data may be encoded with the camera ex 113 or with a serveror the like that performs data transmission. The moving image datacaptured with a camera ex 116 may be transmitted to the streaming serverex 103 via the computer ex 111. The camera ex 116 is a device capable ofcapturing still images and moving images, such as a digital camera. Inthis case, the moving image data may be encoded either with the cameraex 116 or the computer ex 111. The encoding process is executed in anLSI ex 117 included in the computer ex 111 or the camera ex 116. Inaddition, image encoding-decoding software may be installed in a storagemedium (e.g., a CD-ROM, a flexible disk or a hard disk) that is arecording medium readable on the computer ex 111 and the like. Further,the moving image data may be transmitted with the camera-equipped mobilephone ex 115. This moving image data is data encoded in the LSI includedin the mobile phone ex 115. Thus, in the case of applying the presentinvention to a distribution system, an MP4 file that has been stored inadvance is distributed from an HTTP streaming server. That is, an MP4file itself is transmitted, instead of packetizing and distributingencoded data.

Further, the streaming server ex 103 is connected to the camera ex 113via the base station ex 109 and the telephone network ex 104, making itpossible to realize, for example, live distribution based on encodeddata transmitted by the user using the camera ex 113. Additionally, inorder to perform live distribution as described above, it is necessaryto create an MP4 file in real time, so that a fragmented MP4 file isused. By using a fragmented MP4, it is possible to perform low-delaydistribution closer to live distribution, using HTTP/TCP.

In the case of the content serving system ex 100, the user encodes acontent (e.g., a video clip of a music live performance) being capturedwith the camera ex 113, the camera ex 116 and the like and transmits itto the streaming server ex 103, while the streaming server ex 103performs streaming distribution of the above-described content data to arequesting client. Examples of the client include the computer ex 111,the PDA ex 112, the camera ex 113 and the mobile phone ex 114, which arecapable of decoding the encoded data. By doing so, the content servingsystem ex 100 allows the encoded data to be received and reproduced onthe client, and furthermore, it allows the encoded data to be received,decoded and reproduced in real time on the client, thus making itpossible to realize personal broadcasting. Additionally, at the time ofreproducing the content, it is also possible to use a media data displayprogram that realizes, with a computer, the media data display deviceand the media data display method of the above-described embodiment. Forexample, the computer ex 111, the PDA ex 112, the camera ex 113 and themobile phone ex 114 may be provided with a media data display programthat realizes the media data display method described in the aboveembodiment.

As an example, the mobile phone is described.

FIG. 24 is a diagram showing the mobile phone ex 115 using the mediadata display device of the above-described embodiment. The mobile phoneex 115 includes: an antenna ex 201 for transmitting and receiving radiowaves to and from the base station ex 110; a camera portion ex 203, suchas a CCD camera, capable of capturing video and still images; a displayportion ex 202, such as a liquid crystal display, that displays dataobtained by decoding video captured with the camera portion ex 203,video received with the antenna ex 201 and the like; a body portionconstituted by a group of operation keys ex 204; an audio output portionex 208, such as a speaker, for outputting audio; an audio input portionex 205, such as a microphone, for inputting audio; a recording medium ex207 for storing encoded data or decoded data such as data of capturedmoving images or still images, data of received e-mails and data ofmoving images or data of still images; and a slot portion ex 206 forallowing the recording medium ex 207 to be placed in the mobile phone ex115. The recording medium ex 207 stores, in a plastic case such as an SDcard, a flash memory device that is one kind of EEPROM (electricallyerasable and programmable read only memory), which is an electricallyrewritable and erasable nonvolatile memory.

The mobile phone ex 115 is further described with reference to FIG. 25.In the mobile phone ex 115, a power supply circuit portion ex 310, anoperation input control portion ex 304, an image encoding portion ex312, a camera interface portion ex 303, an LCD (liquid crystal display)control portion ex 302, an image decoding portion ex 309, amultiplexing/demultiplexing portion ex 308, a recording/reproducingportion ex 307, a modulating/demodulating circuit portion ex 306 and anaudio processing portion ex 305 are mutually connected via a synchronousbus ex 313 to a main control portion ex 311 that performs centralizedcontrol of various portions of the body portion provided with thedisplay portion ex 202 and the operation keys ex 204.

If an END/Power key is turned on by a user's operation, the power supplycircuit portion ex 310 drives the camera-equipped digital mobile phoneex 115 into an operable state by supplying power to various portionsfrom a battery pack.

In accordance with the control from the main control portion ex 311 madeup of a CPU, a ROM, a RAM and the like, the mobile phone ex 115 convertsan audio signal collected with the audio input portion ex 205 in a voicecall mode into digital audio data with the audio processing portion ex305, and subjects this to spread-spectrum processing with themodulating/demodulating circuit portion ex 306 and to digital-to-analogconversion and frequency conversion with a transmitting/receivingcircuit portion ex 301, and then transmits it via the antenna ex 201.Additionally, the mobile phone ex 115 amplifies a received signalreceived with the antenna ex 201 in a voice call mode, and subjects thisto frequency conversion and analog-to-digital conversion and tospread-spectrum processing with the modulating/demodulating circuitportion ex 306, converts it to an analog audio signal with the audioprocessing portion ex 305, and then outputs it via the audio outputportion ex 208.

Furthermore, in the case of sending an e-mail in a data transmissionmode, text data of the e-mail that has been input by operating theoperation keys ex 204 of the body portion is sent out to the maincontrol portion ex 311 via the operation input control portion ex 304.The main control portion ex 311 subjects the text data tospread-spectrum processing with the modulating/demodulating circuitportion ex 306 and to digital-to-analog conversion and frequencyconversion with the transmitting/receiving circuit portion ex 301, andthen transmits it to the base station ex 110 via the antenna ex 201.

In the case of transmitting image data in a data transmission mode,image data captured with the camera portion ex 203 is supplied to theimage encoding portion ex 312 via the camera interface portion ex 303.On the other hand, in the case of not transmitting image data, imagedata captured with the camera portion ex 203 can also be directlydisplayed in the display portion ex 202 via the camera interface portionex 303 and the LCD control portion ex 302.

The image encoding portion ex 312 converts image data supplied from thecamera portion ex 203 into encoded image data by compressing andencoding it, and sends this to the multiplexing/demultiplexing portionex 308. At the same time, the mobile phone ex 115 sends, in the form ofdigital audio data, audio collected with the audio input portion ex 205during capturing images with the camera portion ex 203 to themultiplexing/demultiplexing portion ex 308 via the audio processingportion ex 305.

The multiplexing/demultiplexing portion ex 308 multiplexes the encodedimage data supplied from the image encoding portion ex 312 and the audiodata supplied from the audio processing portion ex 305 by apredetermined method, and subjects the resulting multiplexed data tospread-spectrum processing with the modulating/demodulating circuitportion ex 306 and to digital-to-analog conversion and frequencyconversion with the transmitting/receiving circuit portion ex 301, andthen transmits it via the antenna ex 201.

In the case of receiving data of a moving image file linked to a website or the like in a data transmission mode, a received signal receivedfrom the base station ex 110 via the antenna ex 201 is subjected tospread-spectrum processing with the modulating/demodulating circuitportion ex 306, and the resulting multiplexed data is sent to themultiplexing/demultiplexing portion ex 308.

In addition, in order to decode the multiplexed data received via theantenna ex 201, the multiplexing/demultiplexing portion ex 308 separatesthe multiplexed data so as to divide it into an encoded bit stream ofimage data and an encoded bit stream of audio data, and supplies theencoded image data to the image decoding portion ex 309 via thesynchronous bus ex 313, and the audio data to the audio processingportion ex 305.

Next, the image decoding portion ex 309 generates reproduction movingimage data by decoding the encoded bit stream of image data, andsupplies this to the display portion ex 202 via the LCD control portionex 302, thereby displaying the moving image data contained in a movingimage file linked to a web site, for example.

In addition, the image decoding portion ex 309 serves the function ofthe media data display device of the above-described embodiment.

At the same time, the audio processing portion ex 305 converts the audiodata into an analog audio signal, and then supplies this to the audiooutput portion ex 208, thereby displaying the audio data contained in amoving image file linked to a web site, for example.

Furthermore, the mobile phone ex 115 may also include a media datadisplay program that realizes the media data display method of theabove-described embodiment. As shown in FIG. 26, the media data displaydevice and the media data display method of the above-describedembodiment, as well as the media data display program that realizes themedia data display method, can also be installed in a reproductiondevice ex 403 that reads and decodes multiplexed data of an encoded bitstream recorded in a storage medium ex 402 such as a CD or a DVD, whichis a recording medium. In this case, the reproduced video signal isdisplayed on a monitor ex 404. Alternatively, it is also possible toinstall the media data display device of the above-described embodimentin a set-top box ex 407 connected to a cable television cable ex 405 orto a satellite/terrestrial broadcasting antenna ex 406, and to reproducethis on a television monitor ex 408. At this time, the media datadisplay device may be incorporated in a television, instead of in theset-top box. It is also possible to receive a signal from a broadcastingsatellite ex 410 or the base station ex 107 with an automobile ex 412having an antenna ex 411, and to reproduce moving images on a displaydevice, such as a car navigation system ex 413 of the automobile ex 412.

Furthermore, it is also possible to encode and multiplex an imagesignal, and to record it in a recording medium. Specific examplesinclude a DVD recorder that records an image signal in a DVD disk ex 421and a recorder ex 420, such as a disk recorder, that records an imagesignal in a hard disk. Additionally, it is also possible to record animage signal in an SD card ex 422. When the recorder ex 420 is providedwith the media data display device of the above-described embodiment, itis possible to reproduce the image signal recorded in the DVD disk ex421 or the SD card ex 422, and to display it on the monitor ex 408.

The car navigation system ex 413 may have, for example, a configurationcorresponding to the configuration shown in FIG. 25 from which thecamera portion ex 203, the camera interface portion ex 303 and the imageencoding portion ex 312 have been omitted, and the same can apply to thecomputer ex 111, a television (receiver) ex 401 and the like.

Further, the terminal such as the above-described mobile phone ex 114can be one of three different types of terminals, namely, atransmitting/receiving terminal having both an encoder and a decoder, areceiving terminal having only an encoder, and a receiving terminalhaving only a decoder.

Thus, it is possible to use the media data display device and the mediadata display method of the above-described embodiment, and the mediadata display program that realizes the media data display method in anyof the above-described devices and systems, and they can achieve theeffect described in the above embodiment.

Other Considerations

Various modifications and combinations can be made to the contentsdescribed above in the first to fourth embodiments, as long as they donot depart from the spirit of the embodiments.

(1)

The display area converting portion 21 and the terminal display screensize acquisition portion 22 described in the first embodiment withreference to FIG. 1 may also be provided in the media data displaydevice 35 described in the second embodiment with reference to FIG. 4.

In this case, the display position determining portion 38 displays apriority display area in a video display area set by the display areaconverting portion 21.

(2)

The display area converting portion 21 and the terminal display screensize acquisition portion 22 described in the first embodiment withreference to FIG. 1 may also be provided in the media data displaydevice 85 or the media data display device 100 described in the thirdembodiment with reference to FIG. 14 or FIG. 21. In this case, thescaling portion 89 or the scaling portion 102 scales frame data withrespect to a video display area set by the display area convertingportion 21. Therefore, both the function of setting the moving imagedisplay area and the scaling function can be realized. Then, when thescaling portion does not perform scaling based on a determination resultfrom the scaling determining means, an effect similar to that of thefirst embodiment can be achieved.

Additionally, the display position determining portion 38 described inthe second embodiment with reference to FIG. 4 may be further provided,and it is possible to let it determine the display position for displaywhen scaling is not performed.

(3)

An MP4 file may occasionally be used for multiplexing and recordingvideo, audio or text data. For example, video and audio captured withthe camera of a mobile phone and the like are recoded as an MP4 file.

At the time of recording an MP4 file, ‘stdc’ and ‘stsl’ described in theabove embodiments can be set in advance by the creator of the MP4 file.Additionally, the display area size (Wp, Hp) of a video display area canbe set during recording video. Here, when the image size (Wv, Hv) ofvideo to be recorded changes, the maximum value of the image size (Wv,Hv) can be set as the display area size (Wp, Hp).

This makes it possible to set a sufficient display area size (Wp, Hp) inthe media data display device at the time of displaying video.

(4)

The pan-scan of MPEG-2 video may be used in combination with the displayof the priority display area using ‘stdc’.

In the pan-scan of MPEG-2 video, an area to be displayed is specified bysetting the difference value between the center of the video image andthe center of the display.

Here, the pan-scan is used for cutting out and displaying only aspecified area of a decoded image frame. That is, areas of the videothat are other than the specified area are not displayed even if thereis a room in the video display area.

On the other hand, ‘stdc’ is aimed at performing a more effectivedisplay in a device having a small display screen and no scalingfunction, by indicating the center of the area to be displayed withpriority. That is, the area to be displayed is not limited, and videocan be displayed in the entire video display area if there is a room inthe video display area.

Therefore, when the pan-scan is combined with the display of thepriority display area using ‘stdc’ and when a specified area cut out bythe pan-scan has a size larger than the display screen, it is possibleto determine which portion of the cut out area has priority, and todisplay the specified area with that portion as the center.

(5)

Although the MP4 file that does not use any fragment is described in theabove-described embodiments, a similar display operation can beperformed for a fragmented MP4 file. At the time of setting displayinformation for each sample in a fragmented MP4 file, the displayinformation is set in each fragment.

(6)

Although a priority area or scaling information is used asdisplay-related information in the above-described embodiments, there isno limitation on using other display-related information. In addition,other display-related information such as setting of display effect maybe used in combination with the above-described display information.

(7)

In the above-described embodiments, it is stated that the priority area,the scaling method and the like may be specified for each chunk.

A case where these are specified for each chunk is described in detailwith reference to FIGS. 27 and 28.

Samples stored in ‘mdat’ are grouped into units called chunks, eachcontaining one or more samples, and information is displayed in units ofchunks by associating the chunks with sample entries. Here, “sampleentry” refers to an entry in ‘stsd’, and stores, for example,initialization information used during decoding a sample. In particular,the sample entry for a video track is called Visual Sample Entry. Inaddition, the association of the chunks and the sample entries isdescribed in ‘stsc’.

The information items (e.g., ‘stdc’ and ‘stsl’) described in the aboveembodiment that specify the priority area, the scaling method and thelike may be stored in Visual Sample Entry in ‘stsd’, instead of directlybelow ‘stbl’. Accordingly, it is possible to switch ‘the display-relatedinformation such as stdc’ and ‘stsl’ for each chunk by referencing‘stsc’ for each chunk.

Here, a syntax of ‘stsl’ stored in Visual Sample Entry of ‘stsd’ isshown in FIG. 27. ‘stsl’ shown in FIG. 27 has a structure similar tothat of the box described with reference to FIG. 29(b), and ischaracterized in that the information items that specify the priorityarea, the scaling method and the like are included in the same box. Inthe following, the content of the syntax is further described.

[constraint_flag] is information for indicating whether to performdisplay in accordance with the scaling method included in ‘stsl’, and isthe same as that described with reference to FIG. 20 in theabove-described embodiment. For example, if the flag is ON (e.g., if thebit has a value of [1]), scaling is performed in the method indicated in[scale_method], which will be described later. On the other hand, if theflag is OFF (e.g., if the bit has a value of [0]), the scaling operationcan be selected at the terminal side, and display may be performedwithout performing scaling. Further, even if display is performed afterscaling, it is not always necessary to follow the method indicated in[scale_method].

[scale_method] is information for indicating the scaling method, and thescaling method is assigned by its value. An example of this assignmentis shown below. For example, value [1] specifies the ‘fill’ mode. Value[2] specifies the ‘hidden’ mode. Value [3] specifies the ‘meet’ mode.Value [4] specifies the ‘slice_x’ mode. Value [5] specifies the ‘slicey’mode.

Here, an 8 bit area is assigned to [scale_method]. This is similar to[scale_method] described in the above embodiment, for example, in thatvalues [128 to 255] can be used as a user specified area.

[display_center_x] and [display_center_y] are information items that arethe same as those described in the above embodiment, and used forspecifying the priority area. Each information item stores value [0] asa default value.

In FIG. 28, the 1st to 30th samples, the 31st to 120th samples, the121st to 180th samples are stored in the first, second and third chunks,respectively. Furthermore, the chunks reference the first, second andthird sample entries, respectively. The sample entries store coordinatesof the center of different priority areas, or different scaling methods,making it possible to switch them for each chunk before performingdisplay.

Additionally, it was stated in the above description that ‘stdc’, ‘stsl’and the like are stored in Visual Sample Entry in ‘stsd’. Here, ‘stdc’,‘stsl’ and the like may be placed directly below ‘stbl’. In this case,information items that associate each of ‘stdc’, ‘stsl’ and the likewith each of the chunks are separately stored. Furthermore, ‘stdc’,‘stsl’ and the like may be associated with chunks, using Sample to GroupBox (‘sbgp’) and Sample Group Description Box (‘sgpd’), which arestandardized in Amendment 1 of ISO Base Format (ISO/IEC 14496-12) or inAVC File Format (ISO/IEC 14496-15). At this time, a type of VisualSample Group Entry that stores information such as ‘stdc’ and ‘stsl isnewly defined.

Appendices

The invention described in the above embodiments can also be expressedas follows.

Content of Appendices

Appendix 1

A media data display device for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,comprising:

a moving image display position determining means for determining amoving image display position, the moving image display position being adisplay position of the moving images; and

a media data display means for displaying the multimedia data, the mediadata display means displaying the moving images at the moving imagedisplay position determined by the moving image display positiondetermining means, without changing their image size.

Appendix 2

The media data display device according to appendix 1,

wherein the multimedia data includes media display position informationindicating a display position of a medium other than the moving imagesin the display screen, and

wherein the moving image display position determining means determinesthe moving image display position in accordance with the media displayposition information.

Appendix 3

The media data display device according to appendix 1, furthercomprising:

a moving image display position setting means capable of setting themoving image display position,

wherein the moving image display position determining means determinesthe moving image display position based on a setting of the moving imageposition setting means.

Appendix 4

The media data display device according to appendix 1,

wherein the multimedia data includes priority display area informationindicating a priority display area, the priority display area beingdisplayed with priority in the moving images, and

wherein the moving image display position determining means determinesthe moving image display position based on the priority display areainformation.

Appendix 5

The media data display device according to appendix 4,

wherein the multimedia data includes moving image display areainformation indicating a moving image display area in which the movingimages can be displayed in the display screen,

wherein the priority display area information is information indicatingthe center of the priority display area, and

wherein the moving image display position determining means determinesthe moving image display position such that the center of the movingimage display area coincides with the center of the priority displayarea, based on the moving image display area information and thepriority display area information.

Appendix 6

The media data display device according to appendix 4 or 5,

wherein the multimedia data includes moving image display areainformation indicating a moving image display area in which the movingimages can be displayed in the display screen, and

wherein the moving image display position determining means determinesthe moving image display position such that the priority display area isincluded and that as large an area as possible of the moving images isdisplayed.

Appendix 7

A media data display device for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,comprising:

a moving image display area setting means for setting, in apredetermined size, a moving image display area in which the movingimages can be displayed in the display screen, when it is determinedthat the moving image display area has an arbitrary size based on movingimage display area information obtained from the multimedia data; and

a media data display means for displaying the multimedia data, the mediadata display means displaying the moving images in the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting means, without changing their image size.

Appendix 8

A media data display device for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,comprising:

a moving image display area setting means for setting, in apredetermined size, a moving image display area in which the movingimages can be displayed in the display screen, when it is determinedthat the moving image display area has a size different from thepredetermined size based on moving image display area informationobtained from the multimedia data; and

a media data display means for displaying the multimedia data, the mediadata display means displaying the moving images in the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting means, without changing their image size.

Appendix 9

A media data display device for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,comprising:

a moving image display area setting means for setting, in apredetermined size, a moving image display area in which the movingimages can be displayed in the display screen, when it is determinedthat the moving image display area has an arbitrary size based on movingimage display area information obtained from the multimedia data;

a scaling means for scaling the moving images with respect to the movingimage display area having the predetermined size set by the moving imagedisplay area setting means, based on scaling specifying information thatspecifies a method for scaling the moving images; and

a media data display means for displaying the multimedia data, the mediadata display means displaying the moving images scaled by the scalingmeans in the moving image display area.

Appendix 10

A media data display device for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,comprising:

a moving image display area setting means for setting, in apredetermined size, a moving image display area in which the movingimages can be displayed in the display screen, when it is determinedthat the moving image display area has a size different from thepredetermined size based on moving image display area informationobtained from the multimedia data;

a scaling means for scaling the moving images with respect to the movingimage display area having the predetermined size set by the moving imagedisplay area setting means, based on scaling specifying information thatspecifies a method for scaling the moving images; and

a media data display means for displaying the multimedia data, the mediadata display means displaying the moving images scaled by the scalingmeans in the moving image display area.

Appendix 11

A media data display method for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,comprising:

a moving image display position determining step of determining a movingimage display position, the moving image display position being adisplay position of the moving images; and

a media data display step of displaying the multimedia data, the mediadata display step displaying the moving images at the moving imagedisplay position determined by the moving image display positiondetermining step, without changing their image size.

Appendix 12

A media data display method for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,comprising:

a moving image display area setting step of setting, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has an arbitrary size based on moving image displayarea information obtained from the multimedia data; and

a media data display step of displaying the multimedia data, the mediadata display step displaying the moving images in the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting step, without changing their image size.

Appendix 13

A media data display method for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,comprising:

a moving image display area setting step of setting, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has a size different from the predetermined sizebased on moving image display area information obtained from themultimedia data; and

a media data display step of displaying the multimedia data, the mediadata display step displaying the moving images in the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting step, without changing their image size.

Appendix 14

A media data display method for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,comprising:

a moving image display area setting step of setting, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has an arbitrary size based on moving image displayarea information obtained from the multimedia data;

a scaling step of scaling the moving images with respect to the movingimage display area having the predetermined size set by the moving imagedisplay area setting step, based on scaling specifying information thatspecifies a method for scaling the moving images; and

a media data display step of displaying the multimedia data, the mediadata display step displaying the moving images scaled by the scalingstep in the moving image display area.

Appendix 15

A media data display method for displaying on a display screen,multimedia data comprising a plurality of media including moving images,comprising:

a moving image display area setting step of setting, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has a size different from the predetermined sizebased on moving image display area information obtained from themultimedia data;

a scaling step of scaling the moving images with respect to the movingimage display area having the predetermined size set by the moving imagedisplay area setting step, based on scaling specifying information thatspecifies a method for scaling the moving images; and

a media data display step of displaying the multimedia data, the mediadata display step displaying the moving images scaled by the scalingstep in the moving image display area.

Appendix 16

A media data display program for performing, with a computer, a mediadata display method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images,

wherein the media data display program lets the computer perform a mediadata display method comprising:

a moving image display position determining step of determining a movingimage display position, the moving image display position being adisplay position of the moving images; and

a media data display step of displaying the multimedia data, the mediadata display step displaying the moving images at the moving imagedisplay position determined by the moving image display positiondetermining step, without changing their image size.

Appendix 17

A media data display program for performing, with a computer, a mediadata display method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images,

wherein the media data display program lets the computer perform a mediadata display method comprising:

a moving image display area setting step of setting, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has an arbitrary size based on moving image displayarea information obtained from the multimedia data; and

a media data display step of displaying the multimedia data, the mediadata display step displaying the moving images in the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting step, without changing their image size.

Appendix 18

A media data display program for performing, with a computer, a mediadata display method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images,

wherein the media data display program lets the computer perform a mediadata display method comprising:

a moving image display area setting step of setting, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has a size different from the predetermined sizebased on moving image display area information obtained from themultimedia data; and

a media data display step of displaying the multimedia data, the mediadata display step displaying the moving images in the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting step, without changing their image size.

Appendix 19

A media data display program for performing, with a computer, a mediadata display method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images,

wherein the media data display program lets the computer perform a mediadata display method comprising:

a moving image display area setting step of setting, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has an arbitrary size based on moving image displayarea information obtained from the multimedia data;

a scaling step of scaling the moving images with respect to the movingimage display area having the predetermined size set by the moving imagedisplay area setting step, based on scaling specifying information thatspecifies a method for scaling the moving images; and

a media data display step of displaying the multimedia data, the mediadata display step displaying the moving images scaled by the scalingstep in the moving image display area.

Appendix 20

A media data display program for performing, with a computer, a mediadata display method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images,

wherein the media data display program lets the computer perform a mediadata display method comprising:

a moving image display area setting step of setting, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has a size different from the predetermined sizebased on moving image display area information obtained from themultimedia data;

a scaling step of scaling the moving images with respect to the movingimage display area having the predetermined size set by the moving imagedisplay area setting step, based on scaling specifying information thatspecifies a method for scaling the moving images; and

a media data display step of displaying the multimedia data, the mediadata display step displaying the moving images scaled by the scalingstep in the moving image display area.

Description of Appendices

The media data display device described in appendix 1 is a media datadisplay device for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises amoving image display position determining means and a media data displaymeans. The moving image display position determining means determines amoving image display position, which is a display position of the movingimages. The media data display means is a means for displaying themultimedia data, and displays the moving images at the moving imagedisplay position determined by the moving image display positiondetermining means, without changing their image size.

Here, “display position” refers to, for example, a position in thedisplay screen when the entire moving image is displayed, and refers toa position of the display area in the moving images when a portion ofthe moving image is displayed (hereinafter, the same applies in thissection). In addition, “multimedia data” refers to, for example, an MP4file in which video, audio, text, still images or the like aremultiplexed and stored (hereinafter, the same applies in this section).

According to the media data display device of the present invention, themedia data display means displays the moving images without changingtheir image size, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. Additionally, the processing load on thedevice can be reduced by displaying the moving images without scalingthem. Furthermore, according to this media data display device, themoving image display position determining means determines the movingimage display position, so that the moving images can be displayed at anappropriate position.

The media data display device described in appendix 2 is the media datadisplay device according to appendix 1, wherein the multimedia dataincludes media display position information indicating a displayposition of a medium other than the moving images in the display screen.The moving image display position determining means determines themoving image display position in accordance with the media displayposition information. Here “medium other than the moving image” refersto, for example, text, still images and the like.

According to the media data display device of the present invention, thedisplay position of the moving images is determined based on the displayposition of a medium other than the moving images, so that the movingimages can be displayed at an appropriate position, with respect to thedisplay position of the medium other than the moving images.

The media data display device described in appendix 3 is the media datadisplay device according to appendix 1, and further comprises a movingimage display position setting means capable of setting the moving imagedisplay position. The moving image display position determining meansdetermines the moving image display position, based on the setting ofthe moving image position setting means.

The media data display device of the present invention further comprisesa moving image display position setting means capable of setting themoving image display position and the moving image display positiondetermining means determines the moving image display position, based onthe setting of the moving image position setting means; therefore, forexample, when the user sets the moving image display positiondetermining means, the moving images can be displayed at a displayposition intended by the user.

The media data display device described in appendix 4 is the media datadisplay device according to appendix 1, wherein the multimedia dataincludes priority display area information indicating a priority displayarea, which is an area displayed with priority in the moving images. Themoving image display position determining means determines the movingimage display position based on the priority display area information.Here, the priority display area information specifies, for example, thecoordinates of the center of the priority display area.

According to the media data display device of the present invention, themultimedia data includes the priority display area information and themoving image display position determining means determines the movingimage display position based on the priority display area information,so that an area of the moving images that is considered as highlysignificant can be displayed.

The media data display device described in appendix 5 is the media datadisplay device according to appendix 4, wherein the multimedia dataincludes moving image display area information indicating a moving imagedisplay area in which the moving images can be displayed in the displayscreen. The priority display area information is information indicatingthe center of the priority display area. The moving image displayposition determining means determines the moving image display positionsuch that the center of the moving image display area coincides with thecenter of the priority display area, based on the moving image displayarea information and the priority display area information.

According to the media data display device of the present invention, thepriority display area information is information indicating the centerof the priority display area, and the moving image display positiondetermining means determines the moving image display position such thatthe center of the moving image display area coincides with the center ofthe priority display area; therefore, an area of the moving images thatis considered as highly significant can be appropriately displayed.

The media data display device of appendix 6 is the media data displaydevice according to appendix 4 or 5, wherein the multimedia dataincludes moving image display area information indicating a moving imagedisplay area in which the moving images can be displayed in the displayscreen. The moving image display position determining means determinesthe moving image display position such that the priority display area isincluded and that as large an area as possible of the moving images isdisplayed.

According to the media data display device of the present invention, themoving image display position determining means determines the movingimage display position such that the priority display area is includedand that as large an area as possible of the moving images is displayed;accordingly, the moving images can be displayed by effectively utilizingthe moving image display area, while reflecting the intention of thecreator of the multimedia data.

The media data display device described in appendix 7 is a media datadisplay device for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises amoving image display area setting means and a media data display means.The moving image display area setting means sets, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has an arbitrary size based on moving image displayarea information obtained from the multimedia data. The media datadisplay means is a means for displaying the multimedia data, anddisplays the moving images in the moving image display area having thepredetermined size set by the moving image display area setting means,without changing their image size. It should be noted that the presentinvention can also be applied to a device having a scaling means andcapable of selecting whether to perform scaling or not.

According to the media data display device of the present invention, themedia data display means displays the moving images without changingtheir image size, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. Additionally, the processing load on thedevice can be reduced by displaying the moving images without scalingthem. Furthermore, since the moving image display area setting meanssets the moving image display area in a predetermined size in apredetermined situation, the moving images can be displayed in a movingimage display area appropriate for the media data display device.

The media data display device described in appendix 8 is a media datadisplay device for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises amoving image display area setting means and a media data display means.The moving image display area setting means sets, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has a size different from the predetermined sizebased on moving image display area information obtained from themultimedia data. The media data display means is a means for displayingthe multimedia data, and displays the moving images in the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting means, without changing their image size. It shouldbe noted that the present invention can also be applied to a devicehaving a scaling means and capable of selecting whether to performscaling or not.

According to the media data display device of the present invention, themedia data display means displays the moving images without changingtheir image size, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. Additionally, the processing load on thedevice can be reduced by displaying the moving images without scalingthem. Furthermore, since the moving image display area setting meanssets the moving image display area in a predetermined size, the movingimages can be appropriately displayed in a moving image display areaused by the user.

The media data display device described in appendix 9 is a media datadisplay device for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises amoving image display area setting means, a scaling means and a mediadata display means. The moving image display area setting means sets, ina predetermined size, a moving image display area in which the movingimages can be displayed in the display screen, when it is determinedthat the moving image display area has an arbitrary size based on movingimage display area information obtained from the multimedia data. Thescaling means scales the moving images with respect to the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting means, based on scaling specifying information thatspecifies a method for scaling the moving images. The media data displaymeans is a means for displaying the multimedia data, and displays themoving images scaled by the scaling means in the moving image displayarea.

According to the media data display device of the present invention, themoving image display area setting means sets, in a predetermined size, amoving image display area in which the moving images can be displayed inthe display screen, when it is determined that the moving image displayarea has an arbitrary size based on moving image display areainformation obtained from the multimedia data, and then the scalingmeans scales the moving images with respect to the moving image displayarea. Thus, the moving image display area setting means sets the movingimage display area in a predetermined size, so that the moving imagescan be appropriately displayed in a moving image display area used bythe user.

The media data display device described in appendix 10 is a media datadisplay device for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises amoving image display area setting means, a scaling means and a mediadata display means. The moving image display area setting means sets, ina predetermined size, a moving image display area in which the movingimages can be displayed in the display screen, when it is determinedthat the moving image display area has a size different from thepredetermined size based on moving image display area informationobtained from the multimedia data. The scaling means scales the movingimages with respect to the moving image display area having thepredetermined size set by the moving image display area setting means,based on scaling specifying information that specifies a method forscaling the moving images. The media data display means is a means fordisplaying the multimedia data, and displays the moving images scaled bythe scaling means in the moving image display area.

According to the media data display device of the present invention, themoving image display area setting means sets, in a predetermined size, amoving image display area in which the moving images can be displayed inthe display screen, when it is determined that the moving image displayarea has a size different from the predetermined size based on movingimage display area information obtained from the multimedia data, andthen the scaling means scales the moving images with respect to themoving image display area. Thus, the moving image display area settingmeans sets the moving image display area in a predetermined size, sothat the moving images can be appropriately displayed in a moving imagedisplay area used by the user.

The media data display method described in appendix 11 is a media datadisplay method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises amoving image display position determining step and a media data displaystep. The moving image display position determining step determines amoving image display position, which is a display position of the movingimages. The media data display step is a step of displaying themultimedia data, and displays the moving images at the moving imagedisplay position determined by the moving image display positiondetermining step, without changing their image size.

According to the media data display method of the present invention, themedia data display step displays the moving images without changingtheir image size, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. Additionally, the processing load can bereduced by displaying the moving images without scaling them.Furthermore, according to this media data display method, the movingimage display position determining step determines the moving imagedisplay position, so that the moving images can be displayed at anappropriate position.

The media data display method described in appendix 12 is a media datadisplay method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises amoving image display area setting step and a media data display step.The moving image display area setting step sets, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has an arbitrary size based on moving image displayarea information obtained from the multimedia data. The media datadisplay step is a step of displaying the multimedia data, and displaysthe moving images in the moving image display area having thepredetermined size set by the moving image display area setting step,without changing their image size.

According to the media data display method of the present invention, themedia data display step displays the moving images without changingtheir image size, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. In addition, the processing load can bereduced by displaying the moving images without scaling them.Furthermore, since the moving image display area setting step sets themoving image display area in a predetermined size in a predeterminedsituation, the moving images can be displayed in an appropriate movingimage display area.

The media data display method described in appendix 13 is a media datadisplay method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises amoving image display area setting step and a media data display step.The moving image display area setting step sets, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has a size different from the predetermined sizebased on moving image display area information obtained from themultimedia data. The media data display step is a step of displaying themultimedia data, and displays the moving images in the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting step, without changing their image size.

According to the media data display method of the present invention, themedia data display step displays the moving images without changingtheir image size, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. In addition, the processing load can bereduced by displaying the moving images without scaling them.Furthermore, since the moving image display area setting step sets themoving image display area in a predetermined size, the moving images canbe appropriately displayed in the moving image display area of a deviceused by the user.

The media data display method described in appendix 14 is a media datadisplay method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, and comprises amoving image display area setting step, a scaling step and a media datadisplay step. The moving image display area setting step sets, in apredetermined size, a moving image display area in which the movingimages can be displayed in the display screen, when it is determinedthat the moving image display area has an arbitrary size based on movingimage display area information obtained from the multimedia data. Thescaling step scales the moving images with respect to the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting step, based on scaling specifying information thatspecifies a method for scaling the moving images. The media data displaystep is a step of displaying the multimedia data, and displays themoving images scaled by the scaling step in the moving image displayarea.

According to the media data display method of the present invention, themoving image display area setting step sets, in a predetermined size, amoving image display area in which the moving images can be displayed inthe display screen, when it is determined that the moving image displayarea has an arbitrary size based on moving image display areainformation obtained from the multimedia data, and then the scaling stepscales the moving images with respect to the moving image display area.Thus, the moving image display area setting step sets the moving imagedisplay area in a predetermined size, so that the moving images can beappropriately displayed in a moving image display area used by the user.

The media data display method described in appendix 15 is a method fordisplaying, on a display screen, multimedia data comprising a pluralityof media including moving images, and comprises a moving image displayarea setting step, a scaling step and a media data display step. Themoving image display area setting step sets, in a predetermined size, amoving image display area in which the moving images can be displayed inthe display screen, when it is determined that the moving image displayarea has a size different from the predetermined size based on movingimage display area information obtained from the multimedia data. Thescaling step scales the moving images with respect to the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting step, based on scaling specifying information thatspecifies a method for scaling the moving images. The media data displaystep is a step of displaying the multimedia data, and displays themoving images scaled by the scaling step in the moving image displayarea.

According to the media data display method of the present invention, themoving image display area setting step sets, in a predetermined size, amoving image display area in which the moving images can be displayed inthe display screen, when it is determined that the moving image displayarea has a size different from the predetermined size based on movingimage display area information obtained from the multimedia data, andthen the scaling step scales the moving images with respect to themoving image display area. Thus, the moving image display area settingstep sets the moving image display area in a predetermined size, so thatthe moving images can be appropriately displayed in a moving imagedisplay area used by the user.

The media data display program described in appendix 16 is a media datadisplay program for performing, with a computer, a media data displaymethod for displaying, on a display screen, multimedia data comprising aplurality of media including moving images, and the media data displayprogram lets the computer perform a media data display method comprisinga moving image display position determining step and a media datadisplay step. The moving image display position determining stepdetermines a moving image display position, which is a display positionof the moving images. The media data display step is a step ofdisplaying the multimedia data, and displays the moving images at themoving image display position determined by the moving image displayposition determining step, without changing their image size.

According to the media data display program of the present invention,the media data display step displays the moving images without changingtheir image size, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. Additionally, the processing load can bereduced by displaying the moving images without scaling them.Furthermore, according to this media data display program, the movingimage display position determining step determines the moving imagedisplay position, so that the moving images can be displayed at anappropriate position.

The media data display program described in appendix 17 is a media datadisplay program for performing, with a computer, a media data displaymethod for displaying, on a display screen, multimedia data comprising aplurality of media including moving images, and the media data displayprogram lets the computer perform a media data display method comprisinga moving image display area setting step and a media data display step.The moving image display area setting step sets, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has an arbitrary size based on moving image displayarea information obtained from the multimedia data. The media datadisplay step is a step of displaying the multimedia data, and displaysthe moving images in the moving image display area having thepredetermined size set by the moving image display area setting step,without changing their image size.

According to the media data display program of the present invention,the media data display step displays the moving images without changingtheir image size, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. Additionally, the processing load can bereduced by displaying the moving images without scaling them.Furthermore, since the moving image display area setting step sets themoving image display area in a predetermined size in a predeterminedsituation, the moving images can be displayed in an appropriate movingimage display area.

The media data display program described in appendix 18 is a media datadisplay program for performing, with a computer, a media data displaymethod for displaying, on a display screen, multimedia data comprising aplurality of media including moving images, and the media data displayprogram lets the computer perform a media data display method comprisinga moving image display area setting step and a media data display step.The moving image display area setting step sets, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has a size different from the predetermined sizebased on moving image display area information obtained from themultimedia data. The media data display step is a step of displaying themultimedia data, and displays the moving images in the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting step, without changing their image size.

According to the media data display program of the present invention,the media data display step displays the moving images without changingtheir image size, so that the moving images are not uniformly scaled anddisplayed in a fixed area, but are displayed in a size intended by thecreator of the multimedia data. Additionally, the processing load can bereduced by displaying the moving images without scaling them.Furthermore, since the moving image display area setting step sets themoving image display area in a predetermined size, the moving images canbe appropriately displayed in the moving image display area of a deviceused by the user.

The media data display program described in appendix 19 is a media datadisplay program for performing, with a computer, a media data displaymethod for displaying, on a display screen, multimedia data comprising aplurality of media including moving images, and the media data displayprogram lets the computer perform a media data display method comprisinga moving image display area setting step, a scaling step and a mediadata display step. The moving image display area setting step sets, in apredetermined size, a moving image display area in which the movingimages can be displayed in the display screen, when it is determinedthat the moving image display area has an arbitrary size based on movingimage display area information obtained from the multimedia data. Thescaling step scales the moving images with respect to the moving imagedisplay area having the predetermined size set by the moving imagedisplay area setting step, based on scaling specifying information thatspecifies a method for scaling the moving images. The media data displaystep is a step of displaying the multimedia data, and displays themoving images scaled by the scaling step in the moving image displayarea.

According to the media data display program of the present invention,the moving image display area setting step sets, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has an arbitrary size based on moving image displayarea information obtained from the multimedia data, and then the scalingstep scales the moving images with respect to the moving image displayarea. Thus, the moving image display area setting step sets the movingimage display area in a predetermined size, so that the moving imagescan be appropriately displayed in a moving image display area used bythe user.

The media data display program described in appendix 20 is a media datadisplay program for performing, with a computer, a media data displaymethod for displaying, on a display screen, multimedia data comprising aplurality of media including moving images, and the media data displayprogram lets the computer perform a media data display method comprisinga moving image display area setting step, a scaling step and a mediadata display step. The moving image display area setting step sets, in apredetermined size, a moving image display area in which the movingimages can be displayed in the display screen, when it is determinedthat the moving image display area has a size different from thepredetermined size based on moving image display area informationobtained from the multimedia data. The scaling step scales the movingimages with respect to the moving image display area having thepredetermined size set by the moving image display area setting step,based on scaling specifying information that specifies a method forscaling the moving images. The media data display step is a step ofdisplaying the multimedia data, and displays the moving images scaled bythe scaling step in the moving image display area.

According to the media data display program of the present invention,the moving image display area setting step sets, in a predeterminedsize, a moving image display area in which the moving images can bedisplayed in the display screen, when it is determined that the movingimage display area has a size different from the predetermined sizebased on moving image display area information obtained from themultimedia data, and then the scaling step scales the moving images withrespect to the moving image display area. Thus, the moving image displayarea setting means sets the moving image display area in a predeterminedsize, so that the moving images can be appropriately displayed in amoving image display area used by the user.

INDUSTRIAL APPLICABILITY

The media data display device, the media data display method and themedia data display program according to the present invention candisplay, on a display screen, multimedia data made up of a plurality ofmedia including moving images in a variety of display methods, so thatthey are useful in the fields described above.

1-14. (canceled)
 15. A media data display device for displaying, on adisplay screen, multimedia data comprising a plurality of mediaincluding moving images, comprising: a scaling determining unit operableto determine whether to display the moving images after changing theirimage size or to display the moving images without changing their imagesize; and a media data display unit operable to display multimedia data,the media data display unit displaying the moving images after switchingwhether to perform scaling or not based on a determination result fromthe scaling determining unit.
 16. The media data display deviceaccording to claim 15, wherein the scaling determining unit makes thedetermination based on scaling determining information included in themultimedia data.
 17. The media data display device according to claim16, wherein the scaling determining information is flag informationindicating whether to perform scaling.
 18. The media data display deviceaccording to claim 15, wherein the scaling determining unit makes thedetermination based on the remaining capacity of a battery with whichthe media data display device is provided.
 19. The media data displaydevice according to claim 15, wherein the scaling determining unitincludes a scaling setting unit operable to set whether to display themoving images after changing their image size or to display the movingimages without changing their image size.
 20. The media data displaydevice according to claim 15, wherein the scaling determining unit makesthe determination based on an attribute of the moving images.
 21. Amedia data display device for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,comprising: a scaling unit operable to scale the moving images withrespect to a moving image display area in which the moving images can bedisplayed in the display screen, based on scaling specifying informationthat specifies a method for scaling the moving images; and a media datadisplay unit operable to display the multimedia data, the media datadisplay unit displaying the moving images scaled by the scaling unit inthe moving image display area.
 22. The media data display deviceaccording to claim 21, wherein the scaling unit obtains the scalingspecifying information from the multimedia data.
 23. The media datadisplay device according to claim 21, further comprising: a scalingdetermining unit operable to determine whether to display the movingimages after changing their image size or to display the moving imageswithout changing their image size, wherein the media data display unitdisplays the moving images after switching whether to perform scaling ornot based on a determination result from the scaling determining unit.24. The media data display device according to claim 22, furthercomprising: a scaling determining unit operable to determine whether todisplay the moving images after changing their image size or to displaythe moving images without changing their image size, wherein the mediadata display unit displays the moving images after switching whether toperform scaling or not based on a determination result from the scalingdetermining unit.
 25. The media data display device according to claim21, further comprising a scaling method setting unit operable to set thescaling specifying information.
 26. A media data display method fordisplaying, on a display screen, multimedia data comprising a pluralityof media including moving images, comprising: a scaling determiningprocess of determining whether to display the moving images afterchanging their image size or to display the moving images withoutchanging their image size; and a media data display process ofdisplaying the multimedia data, the media data display processdisplaying the moving images based on a determination result from thescaling determining process.
 27. A media data display method fordisplaying, on a display screen, multimedia data comprising a pluralityof media including moving images, comprising: a scaling process ofscaling the moving images with respect to a moving image display area inwhich the moving images can be displayed in the display screen, based onscaling specifying information that specifies a method for scaling themoving images; and a media data display process of displaying themultimedia data, the media data display process displaying the movingimages scaled by the scaling process in the moving image display area.28. A media data display program for performing, with a computer, amedia data display method for displaying, on a display screen,multimedia data comprising a plurality of media including moving images,wherein the media data display program lets the computer perform a mediadisplay method comprising: a scaling determining process of determiningwhether to display the moving images after changing their image size orto display the moving images without changing their image size; and amedia data display process of displaying the multimedia data, the mediadata display process displaying the moving images based on adetermination result from the scaling determining process.
 29. A mediadata display program for performing, with a computer, a media datadisplay method for displaying, on a display screen, multimedia datacomprising a plurality of media including moving images, wherein themedia data display program lets the computer perform a media datadisplay method comprising: a scaling process of scaling the movingimages with respect to a moving image display area in which the movingimages can be displayed in the display screen, based on scalingspecifying information that specifies a method for scaling the movingimages; and a media data display process of displaying the multimediadata, the media data display process displaying the moving images scaledby the scaling process in the moving image display area.